Co-Optimization of Capacity and Operation for Battery-Hydrogen Hybrid Energy Storage Systems Based on Deep Reinforcement Learning and Mixed Integer Programming

Tiantian Qian (), Kaifeng Zhang, Difen Shi and Lei Zhang
Additional contact information
Tiantian Qian: School of Electronic Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China
Kaifeng Zhang: School of Automation, Southeast University, Nanjing 210096, China
Difen Shi: School of Electronic Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China
Lei Zhang: School of Electronic Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China

Energies, 2025, vol. 18, issue 21, 1-28

Abstract: The hybrid energy storage system (HESS) that combines battery with hydrogen storage exploits complementary power/energy characteristics, but most studies optimize capacity and operation separately, leading to suboptimal overall performance. To address this issue, this paper proposes a bi-level co-optimization framework that integrates deep reinforcement learning (DRL) and mixed integer programming (MIP). The outer layer employs the TD3 algorithm for capacity configuration, while the inner layer uses the Gurobi solver for optimal operation under constraints. On a standalone PV–wind–load-HESS system, the method attains near-optimal quality at dramatically lower runtime. Relative to GA + Gurobi and PSO + Gurobi, the cost is lower by 4.67% and 1.31%, while requiring only 0.52% and 0.58% of their runtime; compared with a direct Gurobi solve, the cost remains comparable while runtime decreases to 0.07%. Sensitivity analysis further validates the model’s robustness under various cost parameters and renewable energy penetration levels. These results indicate that the proposed DRL–MIP cooperation achieves near-optimal solutions with orders of magnitude speedups. This study provides a new DRL–MIP paradigm for efficiently solving strongly coupled bi-level optimization problems in energy systems.

Keywords: hybrid energy storage system (HESS); hydrogen storage; battery energy storage system; co-optimization; bi-level; deep reinforcement learning (DRL) (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/18/21/5638/pdf (application/pdf)
https://www.mdpi.com/1996-1073/18/21/5638/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:18:y:2025:i:21:p:5638-:d:1780716

Access Statistics for this article

Energies is currently edited by Ms. Cassie Shen

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().