Reliability assessment of multi-agent reinforcement learning algorithms for hybrid local electricity market simulation
Haoyang Zhang,
Dawei Qiu,
Koen Kok and
Nikolaos G. Paterakis
Applied Energy, 2025, vol. 389, issue C, No S0306261925005197
Abstract:
The reliability of data-driven multi-agent reinforcement learning (MARL) algorithms is a critical concern, particularly for complex, large-scale multi-agent decision-making problems. This paper aims to assess the reliability of various MARL algorithms in supporting decision-making for prosumer participants in a hybrid local electricity market (LEM) that combines community-based markets and a peer-to-peer (P2P) market. Specifically, it compares the performance of three MARL algorithms: the multi-agent deep deterministic policy gradient (MADDPG) algorithm and two advanced variants incorporating mean-field approximation and attention mechanisms. To evaluate the reliability of these data-driven MARL algorithms, a model-based bi-level optimization problem is introduced for each agent to assess convergence speed and the proximity of results to the ε-Nash equilibrium, as indicated by the no-regret index. The no-regret index is calculated within a mathematical program with equilibrium constraints (MPEC) by fixing the other agents’ behavior generated from the MARL algorithms. Simulation results demonstrate that the attention-MADDPG algorithm achieves the highest no-regret index (0.81), indicating convergence closest to equilibrium, and the greatest total cost reduction (983€), outperforming the other MARL algorithms. The mean-field-MADDPG algorithm is the most balanced, exhibiting robust convergence with the second-highest no-regret index (0.78) and cost reduction (958.8€) under the lowest computational burden (5.4 seconds per episode).
Keywords: ε-Nash equilibrium; Hybrid local electricity market; Multi-agent reinforcement learning; No-regret index; Reliability (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:
Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261925005197
Full text for ScienceDirect subscribers only
Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
HTML/Text
Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:389:y:2025:i:c:s0306261925005197
Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic
DOI: 10.1016/j.apenergy.2025.125789
Access Statistics for this article
Applied Energy is currently edited by J. Yan
More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().