Application of a Gradient Descent Continuous Actor-Critic Algorithm for Double-Side Day-Ahead Electricity Market Modeling

Huiru Zhao, Yuwei Wang, Sen Guo, Mingrui Zhao and Chao Zhang
Additional contact information
Huiru Zhao: School of Economics and Management, North China Electric Power University, Beijing 102206, China
Yuwei Wang: School of Economics and Management, North China Electric Power University, Beijing 102206, China
Sen Guo: School of Economics and Management, North China Electric Power University, Beijing 102206, China
Mingrui Zhao: School of Economics and Management, North China Electric Power University, Beijing 102206, China
Chao Zhang: School of Economics and Management, North China Electric Power University, Beijing 102206, China

Energies, 2016, vol. 9, issue 9, 1-20

Abstract: An important goal of China’s electric power system reform is to create a double-side day-ahead wholesale electricity market in the future, where the suppliers (represented by GenCOs) and demanders (represented by DisCOs) compete simultaneously with each other in one market. Therefore, modeling and simulating the dynamic bidding process and the equilibrium in the double-side day-ahead electricity market scientifically is not only important to some developed countries, but also to China to provide a bidding decision-making tool to help GenCOs and DisCOs obtain more profits in market competition. Meanwhile, it can also provide an economic analysis tool to help government officials design the proper market mechanisms and policies. The traditional dynamic game model and table-based reinforcement learning algorithm have already been employed in the day-ahead electricity market modeling. However, those models are based on some assumptions, such as taking the probability distribution function of market clearing price ( MCP ) and each rival’s bidding strategy as common knowledge (in dynamic game market models), and assuming the discrete state and action sets of every agent (in table-based reinforcement learning market models), which are no longer applicable in a realistic situation. In this paper, a modified reinforcement learning method, called gradient descent continuous Actor-Critic (GDCAC) algorithm was employed in the double-side day-ahead electricity market modeling and simulation. This algorithm can not only get rid of the abovementioned unrealistic assumptions, but also cope with the Markov decision-making process with continuous state and action sets just like the real electricity market. Meanwhile, the time complexity of our proposed model is only O( n ). The simulation result of employing the proposed model in the double-side day-ahead electricity market shows the superiority of our approach in terms of participant’s profit or social welfare compared with traditional reinforcement learning methods.

Keywords: bidding strategy; double-side day-ahead electricity market; gradient descent continuous Actor-Critic (GDCAC) algorithm; reinforcement learning; market clearing price ( MCP ) (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
https://www.mdpi.com/1996-1073/9/9/725/pdf (application/pdf)
https://www.mdpi.com/1996-1073/9/9/725/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:9:y:2016:i:9:p:725-:d:77836

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().