Integrated energy hub dispatch with a multi-mode CAES–BESS hybrid system: An option-based hierarchical reinforcement learning approach

Feifei Cui, Dou An and Huan Xi

Applied Energy, 2024, vol. 374, issue C, No S0306261924013333

Abstract: The high penetration of renewable energy sources (RES) in power generation has driven demand for advanced integrated energy management systems (IEMS). In this study, to address the challenges of insufficient adaptability to dynamic supply–demand, a multi-type energy IEMS combining compressed air energy storage (CAES) and a battery energy storage system (BESS) is proposed, which operates under a multi-mode energy storage (MES) mechanism with rapid response, long-term balance, and synergic adjustment modes. To address the complexity of sequential decisions, an option-critic based twin delayed deep deterministic policy gradient (OCTD3) algorithm is firstly proposed within the hierarchical reinforcement learning (HRL) framework, enhancing efficiency through encapsulation of subtasks within ”options”. Additionally, model precision is refined by fitting the electricity–gas–heat conversion dynamics of CAES under off-design conditions. Dispatch tasks are modeled as an option-based Semi-Markov Decision Process (SMDP) and optimized by the OCTD3 to improve the power fluctuations index (PFI), comprehensive costs index (CCI), and system response synergy index (SRSI). Comparative simulations reveal that the MES mechanism boosts SRSI by 91.8%, showcasing high adaptability to varied supply–demand scenarios. The OCTD3 algorithm develops five hybrid strategies for CAES–BESS across three modes, effectively cutting costs by reducing electricity purchases and fluctuations expenses, and lowering PFI by 42.2% through balancing peak–valley loads and swiftly responding to transient shifts.

Keywords: Integrated energy management system; Hybrid energy storage; Energy dispatch; Hierarchical reinforcement learning (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261924013333
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:374:y:2024:i:c:s0306261924013333

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2024.123950

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().