Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle

Boyi Xiao, Weiwei Yang, Jiamin Wu, Paul D. Walker and Nong Zhang

Energy, 2022, vol. 253, issue C

Abstract: The modern energy management strategy (EMS) plays a vital role in the energy efficiency of the extended range electric vehicle. However, some modern strategies such as model predictive control (MPC) and dynamic programming (DP) have limited practical potential because they are subject to the pre-known environment information and noise interference. The reinforcement learning (RL)control strategy can be adopted as online control to interact with the vehicle and the environment. In this study, a novel auxiliary power unit (APU) charging strategy with multi-object optimization is proposed to achieve high fuel conversion efficiency while maintaining battery charging health. The state-of-the-art algorithm, Soft Actor-Critic (SAC), is applied to achieve better exploration of the possible APU behaviour and solve the sensitivity and poor convergence problems from the current RL studies. Its performance is further verified by the results of the Deep Deterministic Policy Gradient (DDPG) algorithm and DP. Three innovative targets are selected as the RL rewards for optimization: the engine fuel rate, SOC charging trajectory, and the battery charging rate (C-rate). The first adoption of the battery C-rate monitoring in RL-based energy management strategy helps extend the battery lifespan from excessive discharge. The comparative results show that the SAC had a 36% faster convergence speed than DDPG while providing a smoother and more stable action space. The fuel consumption with SAC also outplays that of DDPG by around 3%, which achieves almost 95% of the global optimization result. The successful deployment of the SAC algorithm as an EMS indicates its standout ability in dealing with wide-range actions and states with high randomness, revealing the practical potential compared with the existing RL strategies.

Keywords: Extended range electric vehicle; Energy management strategy; Deep reinforcement learning; Soft actor-critic; Charge protection (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544222010088
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:253:y:2022:i:c:s0360544222010088

DOI: 10.1016/j.energy.2022.124105

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().