A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids

Ning Wang, Weisheng Xu, Weihui Shao and Zhiyu Xu
Additional contact information
Ning Wang: School of Electronics and Information Engineering, Tongji University, Shanghai 201804, China
Weisheng Xu: School of Electronics and Information Engineering, Tongji University, Shanghai 201804, China
Weihui Shao: Education Technology and Computing Center, Tongji University, Shanghai 200092, China
Zhiyu Xu: School of Electronics and Information Engineering, Tongji University, Shanghai 201804, China

Energies, 2019, vol. 12, issue 15, 1-26

Abstract: Decision-making of microgrids in the condition of a dynamic uncertain bidding environment has always been a significant subject of interest in the context of energy markets. The emerging application of reinforcement learning algorithms in energy markets provides solutions to this problem. In this paper, we investigate the potential of applying a Q-learning algorithm into a continuous double auction mechanism. By choosing a global supply and demand relationship as states and considering both bidding price and quantity as actions, a new Q-learning architecture is proposed to better reflect personalized bidding preferences and response to real-time market conditions. The application of battery energy storage system performs an alternative form of demand response by exerting potential capacity. A Q-cube framework is designed to describe the Q-value distribution iteration. Results from a case study on 14 microgrids in Guizhou Province, China indicate that the proposed Q-cube framework is capable of making rational bidding decisions and raising the microgrids’ profits.

Keywords: microgrids; continuous double auction; Q-learning algorithm; battery energy storage system, Q-cube framework; bidding strategy (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/1996-1073/12/15/2891/pdf (application/pdf)
https://www.mdpi.com/1996-1073/12/15/2891/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:12:y:2019:i:15:p:2891-:d:252180

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().