Safe multi-agent deep reinforcement learning for real-time decentralized control of inverter based renewable energy resources considering communication delay

Guodong Guo, Mengfan Zhang, Yanfeng Gong and Qianwen Xu

Applied Energy, 2023, vol. 349, issue C, No S0306261923010127

Abstract: The increasing penetration of distributed renewable energy resources brings a great challenge for real-time voltage security of distribution grids. The paper proposes a safe multi-agent deep reinforcement learning (MADRL) algorithm for real-time control of inverter-based Volt-Var control (VVC) in distribution grids considering communication delay to minimize the network power loss, while maintaining the nodal voltages in a safe range. The multi-agent VVC is modeled as a constrained Markov game, which is solved by the MADRL algorithm. In the training stage, the safety projection is added to the combined policy to analytically solve an action correction formulation to promote more efficient and safe exploration. In the real-time decision-making stage, a state synchronization block is designed to impute the data under the latest timestamp as the input of the agents deployed in a distributed manner, to avoid instability caused by communication delay. The simulation results show that the proposed algorithm performs well in safe exploration, and also achieves better performance under communication delay.

Keywords: Inverter based renewable energy resources; Distribution grids; Voltage control; Multi-agent reinforcement learning; Safe exploration; Communication delay; Decentralized control (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261923010127
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:349:y:2023:i:c:s0306261923010127

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2023.121648

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().