Multi-Time-Scale Demand Response Optimization in Active Distribution Networks Using Double Deep Q-Networks

Wei Niu, Jifeng Li, Zongle Ma, Wenliang Yin and Liang Feng ()
Additional contact information
Wei Niu: State Grid Liaoning Electric Power Co., Ltd. Dalian Power Supply Company, Dalian 234643, China
Jifeng Li: State Grid Liaoning Electric Power Co., Ltd. Dalian Power Supply Company, Dalian 234643, China
Zongle Ma: State Grid Liaoning Electric Power Co., Ltd. Dalian Power Supply Company, Dalian 234643, China
Wenliang Yin: School of Electrical and Electronic Engineering, Shandong University of Technology, Zhangdian District, Zibo 255000, China
Liang Feng: School of Electrical and Electronic Engineering, Shandong University of Technology, Zhangdian District, Zibo 255000, China

Energies, 2025, vol. 18, issue 18, 1-26

Abstract: This paper presents a deep reinforcement learning-based demand response (DR) optimization framework for active distribution networks under uncertainty and user heterogeneity. The proposed model utilizes a Double Deep Q-Network (Double DQN) to learn adaptive, multi-period DR strategies across residential, commercial, and electric vehicle (EV) participants in a 24 h rolling horizon. By incorporating a structured state representation—including forecasted load, photovoltaic (PV) output, dynamic pricing, historical DR actions, and voltage states—the agent autonomously learns control policies that minimize total operational costs while maintaining grid feasibility and voltage stability. The physical system is modeled via detailed constraints, including power flow balance, voltage magnitude bounds, PV curtailment caps, deferrable load recovery windows, and user-specific availability envelopes. A case study based on a modified IEEE 33-bus distribution network with embedded PV and DR nodes demonstrates the framework’s effectiveness. Simulation results show that the proposed method achieves significant cost savings (up to 35% over baseline), enhances PV absorption, reduces load variance by 42%, and maintains voltage profiles within safe operational thresholds. Training curves confirm smooth Q-value convergence and stable policy performance, while spatiotemporal visualizations reveal interpretable DR behavior aligned with both economic and physical system constraints. This work contributes a scalable, model-free approach for intelligent DR coordination in smart grids, integrating learning-based control with physical grid realism. The modular design allows for future extension to multi-agent systems, storage coordination, and market-integrated DR scheduling. The results position Double DQN as a promising architecture for operational decision-making in AI-enabled distribution networks.

Keywords: demand response; deep reinforcement learning; double DQN; active distribution networks; voltage stability; photovoltaic integration (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/18/18/4795/pdf (application/pdf)
https://www.mdpi.com/1996-1073/18/18/4795/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:18:y:2025:i:18:p:4795-:d:1745480

Access Statistics for this article

Energies is currently edited by Ms. Cassie Shen

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().