Safe multi-agent deep reinforcement learning for decentralized low-carbon operation in active distribution networks and multi-microgrids

Tong Ye, Yuping Huang, Weijia Yang, Guotian Cai, Yuyao Yang and Feng Pan

Applied Energy, 2025, vol. 387, issue C, No S0306261925003393

Abstract: Due to fundamental differences in operational entities between distribution networks and microgrids, the equitable allocation of carbon responsibilities remains challenging. Furthermore, achieving real-time, efficient, and secure low-carbon economic dispatch in decentralized multi-entities continues to face obstacles. Therefore, we propose a co-optimization framework for Active Distribution Networks (ADNs) and multi-Microgrids (MMGs) to improve operational efficiency and reduce carbon emissions through adaptive coordination and decision-making. To facilitate decentralized low-carbon decision-making, we introduce the Spatiotemporal Carbon Intensity Equalization Method (STCIEM). This method ensures privacy and fairness by processing local data and equitably distributing carbon responsibilities. Additionally, we propose a non-cooperative optimization strategy that enables entities to optimize their operations independently while considering both economic and environmental interests. To address the challenges of real-time decision-making and the non-convex nature of low-carbon optimization inherent in traditional approaches, we have developed the Enhanced Action Projection Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (EAP-MATD3) algorithm. This algorithm enhances the actor's objective to address the actor-critic mismatch problem, thereby outperforming conventional safe multi-agent deep reinforcement learning methods by generating optimized actions that adhere to physical system constraints. Experiments conducted on the modified IEEE 33-bus network and IEEE 123-bus network demonstrate the superiority of our approach in effectively balancing economic and environmental objectives within complex energy systems.

Keywords: Active distribution network; Carbon emission allocation; Low-carbon economic operation; Multi-microgrid operation; Safe multi-agent deep reinforcement learning (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261925003393
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:387:y:2025:i:c:s0306261925003393

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2025.125609

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().