Finding Nash equilibrium based on reinforcement learning for bidding strategy and distributed algorithm for ISO in imperfect electricity market

Liying Yu, Peng Wang, Zhe Chen, Dewen Li, Ning Li and Rachid Cherkaoui

Applied Energy, 2023, vol. 350, issue C, No S0306261923010681

Abstract: Finding Nash equilibrium in an imperfect market with active and strategic resources is key to the successful operation of the electricity market. A hierarchical Nash distributed reinforcement learning (HNDRL) framework is proposed to realize the interaction between the market participants and independent system operator (ISO). In the bidding stage, multi-agent Nash policy reinforcement learning is utilized to update the bidding strategies for market participants, i.e., generation company (GenCo) and distributed company (DisCo), which maximizes their own profit with imperfect information. The bidding action is updated by the probability iteration to deal with the continuous bidding strategy. And in the market clearing stage, the distributed dynamic-average consensus optimization algorithm is proposed at the ISO level to obtain the locational marginal price (LMP) and transaction quantities. With the participation of the demand-side in the electricity market, the proposed HNDRL framework can reach a Nash equilibrium with the optimal bidding strategy and handle global resource constraints in a distributed way. In addition, theoretical analysis is provided to ensure that the participants’ strategy exponentially converges to the Nash equilibrium for both convex and non-convex problems. Finally, the IEEE 30-bus system is employed to illustrate the efficiency. Compared with the existing method, the simulation results show that the proposed HNDRL framework has a faster convergence rate with the range of 2.667–3.333 times and can obtain higher profit with the range of 1.4286%–7.1429% at the Nash equilibrium point.

Keywords: Bidding strategy; Imperfect electricity market; Nash equilibrium; Distributed optimization; Reinforcement learning; Multi-agent (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261923010681
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:350:y:2023:i:c:s0306261923010681

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2023.121704

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().