Self-learning energy management strategy for hybrid electric vehicle via curiosity-inspired asynchronous deep reinforcement learning

Jianhao Zhou, Yuan Xue, Da Xu, Chaoxiong Li and Wanzhong Zhao

Energy, 2022, vol. 242, issue C

Abstract: In this study, the formulation of a curiosity-inspired asynchronous advantage actor-critic (A3C+) based model-free, self-learning energy management strategy (EMS) is elaborated for hybrid powertrain. The fuel optimality of standard A3C based EMS is strongly influenced by the target strategy including charge sustenance (CS) and charge depletion (CD), being around 92% and 83% in comparison to dynamic programming (DP) based EMS under training cycle. The corresponding performance of A3C derived CD policy further declined to 75% under testing cycles. Similar tendencies are verified by model predictive control (MPC) and deep deterministic policy gradient (DDPG) based EMSs. To this end, random network distillation (RND) and inverse dynamics model (IDM) techniques are incorporated to form novelty-seeking module (NSM) which is leveraged as intrinsic rewards to facilitate the exploration performance. As a result, the proposed A3C + based EMS obtains significant improvement of global optimality, generalization ability and adaptivity. The fuel optimality of A3C + derived CS and CD policies can be guaranteed by at least 92% and 88% under training and testing cycles, comparing with DP based EMS. Besides, the training and running efficacy of A3C + based EMS are apparently superior to MPC based EMS, demonstrating its realtime implementation potential.

Keywords: Asynchronous deep reinforcement learning; Curiosity-inspired exploration; Energy management strategy; Model predictive control; Random network distillation (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0360544221027973
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:energy:v:242:y:2022:i:c:s0360544221027973

DOI: 10.1016/j.energy.2021.122548

Access Statistics for this article

Energy is currently edited by Henrik Lund and Mark J. Kaiser

More articles in Energy from Elsevier
Bibliographic data for series maintained by Catherine Liu ().