An improved soft actor-critic-based energy management strategy of heavy-duty hybrid electric vehicles with dual-engine system

Dongfang Zhang, Wei Sun, Yuan Zou, Xudong Zhang and Yiwei Zhang

Energy, 2024, vol. 308, issue C

Abstract: While deep reinforcement learning (DRL) based energy management strategies (EMSs) have shown potential for optimizing energy utilization in recent years, challenges such as convergence difficulties and suboptimal control still persist. In this research, a novel DRL algorithm, i.e. an improved soft actor-critic (ISAC) algorithm is applied to the EMS of a heavy-duty hybrid electric vehicles (HDHEV) with dual auxiliary power units (APU) in which the priority experience replay (PER), emphasizing recent experience (ERE) and Muchausen reinforcement learning (MRL) methods are adopted to improve the convergence performance and the HDHEV fuel economy. Simultaneously, a bus voltage calculation model suitable for dual-APUs is proposed and validated using real-world data to ensure the precision of the HDHEV model. Results indicate that the proposed EMS reduces HDHEV fuel consumption by 4.59 % and 2.50 % compared to deep deterministic policy gradient (DDPG) and twin delayed deep deterministic policy gradient (TD3)-based EMSs respectively, narrowing the gap to dynamic programming-based EMS to 7.94 %. The proposed EMS exhibits superior training performance, with a 91.28 % increase in convergence speed compared to other DRL-based EMSs. Furthermore, the ablation experiments also validate the effectiveness of each method in the proposed EMS for the SAC algorithm, further demonstrating its superiority.

Keywords: Heavy-duty hybrid electric vehicles; Energy management strategy; Improved soft actor-critic; Prioritized experience replay; Emphasizing recent experience; Muchausen reinforcement learning (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544224027129
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:308:y:2024:i:c:s0360544224027129

DOI: 10.1016/j.energy.2024.132938

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().