Energy management in HDHEV with dual APUs: Enhancing soft actor-critic using clustered experience replay and multi-dimensional priority sampling

Dongfang Zhang, Wei Sun, Yuan Zou and Xudong Zhang

Energy, 2025, vol. 319, issue C

Abstract: Traditional experience sampling methods in reinforcement learning often overlook sample diversity, which limits learning effectiveness. This research proposes an Enhanced Soft Actor-Critic (ESAC) algorithm for energy management in Heavy-Duty Hybrid Electric Vehicles equipped with dual Auxiliary Power Units. ESAC addresses the limitations of existing methods by integrating multi-dimensional evaluation metrics and the BIRCH clustering algorithm for online experience sampling. The proposed approach optimizes performance in complex multi-power source systems, ensuring diverse sample selection and enhancing learning capacity. Comparative analyses of ESAC against TD3, SAC, and SAC-BIRCH-PER demonstrate that ESAC achieves superior convergence performance, with a nearly 10-episode faster convergence rate than Prioritized Experience Replay. Additionally, ESAC shows significant reductions in fuel consumption—up to 5.32 % compared to the dynamic programming benchmark—outperforming SAC and TD3 by 10.54 % and 8.84 %, respectively. These results highlight that enhancing data diversity and prioritization not only stabilizes learning but also optimizes fuel efficiency in low-speed, high-torque conditions, thereby providing a robust solution for real-world energy management challenges.

Keywords: Heavy-duty hybrid electric vehicles; Energy management strategy; Soft actor-critic; BIRCH algorithm; Multi-dimensions priority (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544225005687
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:319:y:2025:i:c:s0360544225005687

DOI: 10.1016/j.energy.2025.134926

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().