An indirect reinforcement learning based real-time energy management strategy via high-order Markov Chain model for a hybrid electric vehicle

Ningkang Yang, Lijin Han, Changle Xiang, Hui Liu and Xunmin Li

Energy, 2021, vol. 236, issue C

Abstract: This paper proposes a real-time indirect reinforcement learning based strategy to reduce the fuel consumption. In order to improve the real-time performance and achieve learning online, the simulated experience from environment model is adopted for the learning process, which is called indirect reinforcement learning. To establish an accurate environment model, a high-order Markov Chain is introduced and detailed, which is more precise than a widely used first-order Markov Chain. Corresponding with the model, how the reinforcement learning algorithm learns from the simulated experience is illustrated. Furthermore, an online recursive form of the transition probability matrix is derived, through which the statistical characteristics from the practical driving conditions can be collected. The induced matrix norm is chosen as a criterion to quantify the differences between the transition probability matrices and to determine the time for updating the environment model and triggering the recalculation of the reinforcement learning algorithm. Simulation results demonstrate that, compared with the direct RL, the proposed strategy can effectively reduce the learning time while maintains satisfied fuel economy. Furthermore, a hardware-in-the-loop experiment verifies its real-time capability and actual applicability.

Keywords: Hybrid electric vehicle; Real-time energy management; Indirect reinforcement learning; High-order Markov chain (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544221015851
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:236:y:2021:i:c:s0360544221015851

DOI: 10.1016/j.energy.2021.121337

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().