An apprenticeship-reinforcement learning scheme based on expert demonstrations for energy management strategy of hybrid electric vehicles

Dong Hu, Hui Xie, Kang Song, Yuanyuan Zhang and Long Yan

Applied Energy, 2023, vol. 342, issue C, No S0306261923005913

Abstract: Deep reinforcement learning (DRL) is a potential solution to develop efficient energy management strategies (EMS) for hybrid electric vehicles (HEV) that can adapt to the changing topology of electrified powertrains and the uncertainty of various driving scenarios. However, traditional DRL has many disadvantages, such as low efficiency and poor stability. This study proposes an apprenticeship-reinforcement learning (A-RL) framework based on expert demonstration (ED) model embedding to improve DRL. First, the demonstration data, calculated by dynamic programming (DP), were collected, and domain adaptive meta-learning (DAML) was used to train the ED model with the adaptive capability of working conditions. Then combined apprenticeship learning (AL) with DRL, and the ED model was used to guide the DRL to output action. The method was validated on three HEV models, and the results show that the training convergence rate increases significantly under the framework. The average increase that the apprenticeship-deep deterministic policy gradient (A-DDPG) based method applied to three HEVs achieved was 34.9 %. Apprenticeship-twin delayed twin delayed deep deterministic policy gradient (A-TD3) achieved 23 % acceleration in the power-split HEV. Because A-DDPG's EMS is more forward-looking and can mimic ED to some extent, the frequency of engine operation in the high-efficiency range has increased. Therefore, A-DDPG can improve the fuel economy of the series hybrid electric bus (HEB) by 0.2–2.7 %, and improvements averaged to about 9.6 % in the series–parallel HEV while maintaining the final SOC. This study aims to improve the sampling efficiency and optimal performance of EMS-based DRL and provide a basis for the design and development of vehicle energy saving and emission reduction.

Keywords: Deep reinforcement learning; Energy management strategy; Hybrid electric vehicles; Apprenticeship learning; Meta-learning (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261923005913
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:342:y:2023:i:c:s0306261923005913

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2023.121227

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().