Design and Improvement of SD3-Based Energy Management Strategy for a Hybrid Electric Urban Bus

Kunyu Wang, Rong Yang (), Yongjian Zhou, Wei Huang () and Song Zhang
Additional contact information
Kunyu Wang: College of Mechanical Engineering, Guangxi University, Nanning 530004, China
Rong Yang: College of Mechanical Engineering, Guangxi University, Nanning 530004, China
Yongjian Zhou: College of Mechanical Engineering, Guangxi University, Nanning 530004, China
Wei Huang: College of Mechanical Engineering, Guangxi University, Nanning 530004, China
Song Zhang: Guangxi Yuchai Machinery Company Limited, Yulin 537000, China

Energies, 2022, vol. 15, issue 16, 1-21

Abstract: With the rapid development of machine learning, deep reinforcement learning (DRL) algorithms have recently been widely used for energy management in hybrid electric urban buses (HEUBs). However, the current DRL-based strategies suffer from insufficient constraint capability, slow learning speed, and unstable convergence. In this study, a state-of-the-art continuous control DRL algorithm, softmax deep double deterministic policy gradients (SD3), is used to develop the energy management system of a power-split HEUB. In particular, an action masking (AM) technique that does not alter the SD3′s underlying principles is proposed to prevent the SD3-based strategy from outputting invalid actions that violate the system’s physical constraints. Additionally, the transfer learning (TL) method of the SD3-based strategy is explored to avoid repetitive training of neural networks in different driving cycles. The results demonstrate that the learning performance and learning stability of SD3 are unaffected by AM and that SD3 with AM achieves control performance that is highly comparable to dynamic planning for both the CHTC-B and WVUCITY driving cycles. Aside from that, TL contributes to the rapid development of SD3. TL can speed up SD3’s convergence by at least 67.61% without significantly affecting fuel economy.

Keywords: hybrid electric urban bus; energy management strategy; deep reinforcement learning; action masking (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/15/16/5878/pdf (application/pdf)
https://www.mdpi.com/1996-1073/15/16/5878/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:15:y:2022:i:16:p:5878-:d:887478

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().