Integrated three-stage decentralized scheduling for virtual power plants: A model-assisted multi-agent reinforcement learning method

Biao Xu, Wenpeng Luan, Jing Yang, Bochao Zhao, Chao Long, Qian Ai and Jiani Xiang

Applied Energy, 2024, vol. 376, issue PA, No S0306261924013680

Abstract: Virtual power plant (VPP) emerges as a promising integration and aggregation technology that facilitates the utilization of massive flexible demand-side resources (DSRs). However, non-negligible modeling errors and high-dimensional uncertainties involved in DSR aggregation threaten the delivery reliability and cost-effectiveness of VPP operation. To address this problem, this study proposes an integrated three-stage scheduling framework for VPPs and develops a model-assisted multi-agent reinforcement learning (MARL) approach. In the proposed framework, the VPP scheduling problem is formulated as a decentralized partially observable Markov Decision Process (Dec-POMDP), which depicts the complex interaction process among the three stages (bidding, re-dispatching and disaggregation). The interactions are evaluated by a comprehensive reward function, incorporating the trading and operation costs, as well as imbalance penalties. To enable decentralized decision-making, a model-assisted multi-agent proximal policy optimization (MA2PPO) algorithm is proposed, which trains a separate actor network for each aggregator. Additionally, the MA2PPO is augmented with a model-assisted safety decision-making method to accelerate the training process. Numerical simulation results verify that the proposed method enhances the delivery reliability and cost-effectiveness of the VPP, while achieving faster convergence time compared with purely model-free MARL methods.

Keywords: Integrated three-stage decentralized scheduling; demand-side resource; intraday market; multi-agent reinforcement learning; virtual power plant (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261924013680
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:376:y:2024:i:pa:s0306261924013680

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2024.123985

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().