Energy management in integrated energy system with electric vehicles as mobile energy storage: An approach using bi-level deep reinforcement learning

Longxiang Chen, Huan He, Rui Jing, Meina Xie and Kai Ye

Energy, 2024, vol. 307, issue C

Abstract: The integrated energy system with electric vehicle charging station via vehicle-to-grid aims to offer a proactive solution for low-carbon development of both energy and transportation sectors. However, achieving optimal energy efficiency with minimal operational costs in such a complex system is challenging due to the high randomness of electric vehicle travel patterns. This work proposes a reinforcement learning-based energy management framework to optimize the coordinated scheduling of integrated energy and vehicle-to-grid. The problem is modeled as a Markov Decision Process and optimized using the bi-level soft actor-critic algorithm, where the upper-level agent supervises the scheduling of the integrated energy system, and lower-level agent sets the electric vehicles’ charging and discharging strategies based on upper-level decision. Numerical simulations demonstrated that by adopting a bi-level reinforcement learning approach, the proposed algorithm effectively enhances energy exchange between integrated energy and electric vehicle charging station, reducing operational costs by 8 % compared to other multi-agent algorithms. Furthermore, when compared to the Mixed-Integer Linear Programming model, the proposed algorithm achieves similar optimality with significantly less computation time, maintaining a margin of error of approximately 4 %. Additionally, integrating electric vehicles as mobile energy storage within this framework can lead to a further 10 % reduction in operating costs.

Keywords: Integrated energy system; Electric vehicle; Deep reinforcement learning; Charging scheduling (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544224025313
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:307:y:2024:i:c:s0360544224025313

DOI: 10.1016/j.energy.2024.132757

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().