Multi-step reinforcement learning for model-free predictive energy management of an electrified off-highway vehicle

Quan Zhou, Ji Li, Bin Shuai, Huw Williams, Yinglong He, Ziyang Li, Hongming Xu and Fuwu Yan

Applied Energy, 2019, vol. 255, issue C

Abstract: The energy management system of an electrified vehicle is one of the most important supervisory control systems which manages the use of on-board energy resources. This paper researches a ‘model-free’ predictive energy management system for a connected electrified off-highway vehicle. A new reinforcement learning algorithm with the capability of ‘multi-step’ learning is proposed to enable the all-life-long online optimisation of the energy management control policy. Three multi-step learning strategies (Sum-to-Terminal, Average-to-Neighbour Recurrent-to-Terminal) are researched for the first time. Hardware-in-the-loop tests are carried out to examine the control functionality for real application of the proposed ‘model-free’ method. The results show that the proposed method can continuously improve the vehicle’s energy efficiency during the real-time hardware-in-the-loop test, which increased from the initial level of 34% to 44% after 5 h’ 35-step learning. Compared with a well-designed model-based predictive energy management control policy, the model-free predictive energy management method can increase the prediction horizon length by 71% (from 35 to 65 steps with 1 s interval in real-time computation) and can save energy by at least 7.8% for the same driving conditions.

Keywords: Model-free predictive control; Energy management; Multi-step reinforcement learning; Markov decision problem; Hybrid electric vehicle (search for similar items in EconPapers)
Date: 2019
References: Add references at CitEc
Citations: View citations in EconPapers (29)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0306261919314424
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:appene:v:255:y:2019:i:c:s0306261919314424

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/405891/bibliographic
http://www.elsevier. ... 405891/bibliographic

DOI: 10.1016/j.apenergy.2019.113755

Access Statistics for this article

Applied Energy is currently edited by J. Yan

More articles in Applied Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().