Enhanced Dynamic Expansion Planning Model Incorporating Q-Learning and Distributionally Robust Optimization for Resilient and Cost-Efficient Distribution Networks

Gang Lu, Bo Yuan, Baorui Nie (), Peng Xia, Cong Wu and Guangzeng Sun
Additional contact information
Gang Lu: State Grid Energy Research Institute Co., Ltd., Beijing 102209, China
Bo Yuan: State Grid Energy Research Institute Co., Ltd., Beijing 102209, China
Baorui Nie: School of Economics and Management, North China Electric Power University, Beijing 102206, China
Peng Xia: State Grid Energy Research Institute Co., Ltd., Beijing 102209, China
Cong Wu: State Grid Energy Research Institute Co., Ltd., Beijing 102209, China
Guangzeng Sun: State Grid Energy Research Institute Co., Ltd., Beijing 102209, China

Energies, 2025, vol. 18, issue 5, 1-25

Abstract: The increasing integration of renewable energy-based distributed generation (DG) in modern distribution networks is essential for reducing reliance on fossil fuels. However, the unpredictability and intermittency of renewable sources such as wind and photovoltaic (PV) systems introduce significant challenges for distribution network planning. To address these challenges, this paper proposes a Q-learning-based Distributionally Robust Optimization (DRO) model for expansion planning of distribution networks and generation units. The proposed model incorporates energy storage systems (ESSs), renewable DG, substations, and distribution lines while considering uncertainties such as renewable generation variability, load fluctuations, and system contingencies. Through a dynamic decision-making process using Q-learning, the model adapts to changing network conditions to minimize the total system cost while maintaining reliability. The Latin Hypercube Sampling (LHS) method is employed to generate multi-scenario data, and piecewise linearization is used to reduce the computational complexity of the AC power flow equations. Numerical results demonstrate that the model significantly improves system reliability and economic efficiency under multiple uncertainty scenarios. The results also highlight the crucial role of the ESS in mitigating the variability of renewable energy and reducing the expected energy not supplied (EENS).

Keywords: renewable energy integration; distribution network expansion; energy storage systems; Q-learning; distributionally robust optimization; system reliability; expected energy not supplied (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/18/5/1020/pdf (application/pdf)
https://www.mdpi.com/1996-1073/18/5/1020/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:18:y:2025:i:5:p:1020-:d:1595435

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().