Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning

Lijin Han, Ke Yang, Tian Ma, Ningkang Yang, Hui Liu and Lingxiong Guo

Energy, 2022, vol. 259, issue C

Abstract: Hybrid electric vehicles (HEVs) bridge the gap between internal combustion engine vehicles and pure electric vehicles, and are therefore regarded as a promising solution to the energy crisis. This paper proposes a real-time energy management strategy (EMS) for hybrid electric vehicles based on reinforcement learning (RL) to improve fuel economy and minimize battery degradation. First, an online recursive Markov chain (MC) is developed that continuously collects statistical features from actual driving conditions, and thus an adaptive and accurate environment model is established. Then, a novel RL algorithm, eligibility trace, is introduced to learn the control policy online based on MC model. By introducing a trace-decay parameter, the eligibility trace algorithm unifies the returns of different steps, forming a more reliable estimate of the optimal value function, and therefore outperforms traditional RL algorithms in optimization. Furthermore, induced matrix norm (IMN) is employed as a standard to measure difference between transition probability matrices (TPM) of MC and to decide when to update environment model as well as recalculate the control policy. Therefore, the EMS's adaptability to various driving conditions are significantly enhanced. Simulation results indicate that eligibility trace shows the best performance in both improving fuel economy and reducing battery life loss compared with Q-learning and rule-based method.

Keywords: Real-time energy management strategy; Hybrid electric vehicle; Reinforcement learning; Eligibility trace; Battery life (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544222018849
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:259:y:2022:i:c:s0360544222018849

DOI: 10.1016/j.energy.2022.124986

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().