The application of machine learning based energy management strategy in multi-mode plug-in hybrid electric vehicle, part I: Twin Delayed Deep Deterministic Policy Gradient algorithm design for hybrid mode

Changcheng Wu, Jiageng Ruan, Hanghang Cui, Bin Zhang, Tongyang Li and Kaixuan Zhang

Energy, 2023, vol. 262, issue PB

Abstract: As the performance of Energy Management Strategy (EMS) is crucial for the energy efficiency of Hybrid Electric Vehicles (HEVs), a Deep Reinforcement Learning (DRL)-based algorithm, namely Twin Delayed Deep Deterministic Policy Gradient (TD3), is adopted to design EMS for the power Charge-Sustained (CS) stage of a multi-mode plug-in Hybrid Electric Vehicle (HEV). In addition, EMS is improved by combining the actor-network of TD3 with Gumbel-Softmax to realize mode selection and torque distribution simultaneously, which is a discrete (mode)-continuous (engine speed) hybrid action space and not applicable in original TD3. To reduce the unreasonable exploration of agents in discrete action, a rule-based mode control mechanism (RBMCM) is designed and involved in EMS. The improved algorithm speeds up the learning process and achieves better fuel economy. Simulation results show that the gap between the proposed strategy and the benchmark dynamic programming (DP) is reduced to 2.55% in the selected training cycle. Regarding the unknown testing cycles, the fuel economy of agents trained by the improved method overperforms traditional DRL-based EMS when it reaches more than 90% of the DP-based benchmarking. In conclusion, the proposed method provides a theoretical foundation for the solution of the hybrid space optimization problem for hybrid systems.

Keywords: TD3; DDPG; Gumbel-softmax; Energy management strategy; Mode control mechanism (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S036054422201979X
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:262:y:2023:i:pb:s036054422201979x

DOI: 10.1016/j.energy.2022.125084

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().