Real-time dispatch of an integrated energy system based on multi-stage reinforcement learning with an improved action-choosing strategy

Lingwei Zheng, Hao Wu, Siqi Guo and Xinyu Sun

Energy, 2023, vol. 277, issue C

Abstract: The uncertainty of renewable energy has brought challenges to the real-time dispatch of integrated energy systems (IES). Nowadays, reinforcement learning (RL) is widely used in IES real-time dispatch to deal with the uncertainty of renewable energy. However, traditional RL algorithms often face the problem of dimensionality when there are numerous controllable units in the system, which will increase operating costs and training time significantly. Based on the above issues, we developed a novel real-time dispatch method for IES with RL model training in stages based on the dueling double deep quality network (D3QN). Dispatches of different controllable units in IES are decomposed into a multi-stage training process according to the degree of thermoelectric coupling and the complexity of equipment operation. This makes the action space of each training stage independent, alleviating the problem of extra-large action space in the traditional RL method. In addition, an improved action-choosing strategy is proposed to enhance local optimization in the process of algorithm training by introducing “offset” according to probability in the training progress. The simulations were carried out on four different types of days in an IES. The results show that the proposed method can effectively reduce operating costs and accelerate convergence.

Keywords: Integrated energy system; Real-time dispatch; Uncertainty; Multi-stage reinforcement learning; Improved action-choosing strategy (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544223010307
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:277:y:2023:i:c:s0360544223010307

DOI: 10.1016/j.energy.2023.127636

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().