Optimal scheduling of electric-gas-thermal-hydrogen integrated energy system considering uncertainties and safe guarantee: A TD3-MIP-based approach

Ying Lei, Liyuan Zhao, Junhua Gu and Jingshu Wang

Energy, 2025, vol. 332, issue C

Abstract: The multiple uncertainties arising from sources and loads present substantial challenges to maintaining the supply-demand balance in the integrated energy system (IES). Data-driven deep reinforcement learning (DRL) algorithms have gained attention for handling complex scheduling problems due to their model-free capabilities. However, ensuring safety remains a significant issue for DRL algorithms, as the agent may fail to satisfy all operation constraints simultaneously. To address these challenges, this paper proposes an IES scheduling approach that combines twin delayed deep deterministic policy gradient (TD3) and mixed-integer programming (MIP), termed as TD3-MIP, which considers uncertainties and provides a safety guarantee. Firstly, to improve energy efficiency and promote low-carbon energy transitions, hydrogen energy is introduced into the IES. A framework for joint electric-gas-thermal-hydrogen scheduling is designed, and the scheduling problem is expressed as a reinforcement learning model. Secondly, the TD3 algorithm is employed to train the scheduling model, dealing with the uncertainties in sources and loads. The operation costs, unbalance penalties and out-of-limit penalties are integrated into the reward function to guide the agent in learning safe and cost-effective decisions during the offline training process. Furthermore, a novel safety IES scheduling strategy is proposed to enforce all operation constraints and achieve supply-demand balance rigorously during the online scheduling process. In this strategy, the trained TD3 critic network structure is redefined as an MIP formulation by introducing a binary variable. In addition, all of the operation constraints are added to the MIP model, which is then solved using a mathematical solver. Finally, simulation results under various scenarios demonstrate that the proposed TD3-MIP approach effectively deals with uncertainties in renewable energy generation and demands under strict constraints. It also shows a competitive edge in cost-efficiency compared to other scheduling approaches.

Keywords: Integrated energy system; Deep reinforcement learning; Safe guarantee; Mixed-integer programming; Multiple uncertainties (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544225026933
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:332:y:2025:i:c:s0360544225026933

DOI: 10.1016/j.energy.2025.137051

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().