Privacy-preserving multi-level co-regulation of VPPs via hierarchical safe deep reinforcement learning

Lin Xue, Yao Zhang, Jianxue Wang, Haotian Li and Fangshi Li

Applied Energy, 2024, vol. 371, issue C, No S0306261924010377

Abstract: Large amounts of distributed energy sources have brought challenges in terms of the safe and stable operation of the power grid. As a key technology between user-side energy resources and the distribution network (DN), how to realize the online coordinated scheduling of VPP and DN as well as the real-time response strategy of distributed equipment (DE) within VPP is the focus of this study. Thus, the hierarchical deep reinforcement learning (DRL) Hierarchical-TD3 algorithm is designed based on the unified modeling of adjustable space to achieve the real-time economic scheduling of VPPs. The upper layer DN considers the network security constraints and solves the economic scheduling model of VPPs based on the single-agent TD3 algorithm. Based on the scheduling instructions from the upper layer, the lower layer VPPs consider the requirements of privacy protection and control autonomy and realize real-time response of the DE within VPP via multi-agent MATD3 algorithm. Numerical results in the modified 33-nodes system show that the proposed Hierarchical-TD3 algorithm can achieve privacy protection and the coordinated scheduling of VPP and DE to reduce the operating cost. It differs from the optimal value by only 1.46% but can achieve online decision-making on the millisecond scale. Compared with the traditional centralized and decentralized DRL algorithms, the total cost is reduced by 10.15% and 5.52% respectively. Compared with the traditional soft-constraint method, there is no constraint violation during the training and testing phases. Finally, the actual 116-nodes testing system validates the scalability of the proposed Hierarchical-TD3 algorithm.

Keywords: Adjustable space unified modeling; Centralized-decentralized coordinated schedule; Hierarchical deep reinforcement learning; Multiple virtual power plants; Privacy protection; Safe exploration and training (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261924010377
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:371:y:2024:i:c:s0306261924010377

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2024.123654

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().