EconPapers    
Economics at your fingertips  
 

Reliability assessment of multi-agent reinforcement learning algorithms for hybrid local electricity market simulation

Haoyang Zhang, Dawei Qiu, Koen Kok and Nikolaos G. Paterakis

Applied Energy, 2025, vol. 389, issue C, No S0306261925005197

Abstract: The reliability of data-driven multi-agent reinforcement learning (MARL) algorithms is a critical concern, particularly for complex, large-scale multi-agent decision-making problems. This paper aims to assess the reliability of various MARL algorithms in supporting decision-making for prosumer participants in a hybrid local electricity market (LEM) that combines community-based markets and a peer-to-peer (P2P) market. Specifically, it compares the performance of three MARL algorithms: the multi-agent deep deterministic policy gradient (MADDPG) algorithm and two advanced variants incorporating mean-field approximation and attention mechanisms. To evaluate the reliability of these data-driven MARL algorithms, a model-based bi-level optimization problem is introduced for each agent to assess convergence speed and the proximity of results to the ε-Nash equilibrium, as indicated by the no-regret index. The no-regret index is calculated within a mathematical program with equilibrium constraints (MPEC) by fixing the other agents’ behavior generated from the MARL algorithms. Simulation results demonstrate that the attention-MADDPG algorithm achieves the highest no-regret index (0.81), indicating convergence closest to equilibrium, and the greatest total cost reduction (983€), outperforming the other MARL algorithms. The mean-field-MADDPG algorithm is the most balanced, exhibiting robust convergence with the second-highest no-regret index (0.78) and cost reduction (958.8€) under the lowest computational burden (5.4 seconds per episode).

Keywords: ε-Nash equilibrium; Hybrid local electricity market; Multi-agent reinforcement learning; No-regret index; Reliability (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261925005197
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:389:y:2025:i:c:s0306261925005197

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2025.125789

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().

 
Page updated 2025-04-30
Handle: RePEc:eee:appene:v:389:y:2025:i:c:s0306261925005197