Reinforcement Learning-Based Lane Change Decision for CAVs in Mixed Traffic Flow under Low Visibility Conditions

Bowen Gong, Zhipeng Xu, Ruixin Wei, Tao Wang (), Ciyun Lin () and Peng Gao
Additional contact information
Bowen Gong: Department of Traffic Information and Control Engineering, Jilin University, Changchun 130022, China
Zhipeng Xu: Department of Traffic Information and Control Engineering, Jilin University, Changchun 130022, China
Ruixin Wei: Department of Traffic Information and Control Engineering, Jilin University, Changchun 130022, China
Tao Wang: China Academy of Transportation Sciences, Beijing 100029, China
Ciyun Lin: Department of Traffic Information and Control Engineering, Jilin University, Changchun 130022, China
Peng Gao: Qingdao Transportation Public Service Center, Qingdao Municipal Transport Bureau, Qingdao 266061, China

Mathematics, 2023, vol. 11, issue 6, 1-24

Abstract: As an important stage in the development of autonomous driving, mixed traffic conditions, consisting of connected autonomous vehicles (CAVs) and human-driven vehicles (HDVs), have attracted more and more attention. In fact, the randomness of human-driven vehicles (HDV) is the largest challenge for connected autonomous vehicles (CAV) to make reasonable decisions, especially in lane change scenarios. In this paper, we propose the problem of lane change decisions for CAV in low visibility and mixed traffic conditions for the first time. First, we consider the randomness of HDV in this environment and construct a finite state machine (FSM) model. Then, this study develops a partially observed Markov decision process (POMDP) for describing the problem of lane change. In addition, we use the modified deep deterministic policy gradient (DDPG) to solve the problem and get the optimal lane change decision in this environment. The reward designing takes the comfort, safety and efficiency of the vehicle into account, and the introduction of transfer learning accelerates the adaptation of CAV to the randomness of HDV. Finally, numerical experiments are conducted. The results show that, compared with the original DDPG, the modified DDPG has a faster convergence velocity. The strategy learned by the modified DDPG can complete the lane change in most of the scenarios. The comparison between the modified DDPG and the rule-based decisions indicates that the modified DDPG has a stronger adaptability to this special environment and can grasp more lane change opportunities.

Keywords: reinforcement learning; low visibility and mixed traffic conditions; lane change decision; DDPG (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/6/1556/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/6/1556/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:6:p:1556-:d:1104571

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().