Hierarchical reinforcement learning based energy management strategy of plug-in hybrid electric vehicle for ecological car-following process

Hailong Zhang, Jiankun Peng, Hanxuan Dong, Huachun Tan and Fan Ding

Applied Energy, 2023, vol. 333, issue C, No S0306261922018566

Abstract: The economy-oriented automated hybrid eclectic vehicles (HEV) provide great potential to save energy by optimizing both driving behaviors and power distribution. Recent advances in the ecological car following issue of HEV focus on fusing adaptive cruise control (ACC) and energy management system (EMS) by collaborative optimization. However, series control frameworks ACC+EMS breaks the internal coupling relation between motion control and energy distribution, leading to the natural limitation of its optimization. On the opposite, integrated ACC-EMS promises energy-saving improvement but brings complex optimization problems with multi-scale objectives and large exploration space. The huge computation load restricts the online application of ACC-EMS. To address these problems, a hierarchical reinforcement learning based ACC-EMS strategy is proposed with a hierarchical policy and non-hierarchical execution. The upper layer learns to plan state-of-charge and time-headway trajectories, while the low layer policy learns to achieve the expected goals by outputting control variables executed by the host vehicle. The proposed ACC-EMS strategy were self-learning by interaction in car-following scenario constructed with GPS data on I-880 highway. Comprehensive simulations show the proposed strategy has significantly improved the training speed and stability, compared to the offline global optimum, achieving the energy consumption difference of less than 3% and computational load of less than 600 times.

Keywords: Hybrid electric vehicle; Reinforcement learning; Adaptive cruise control; Energy management (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261922018566
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:333:y:2023:i:c:s0306261922018566

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2022.120599

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().