Multi-Vehicle Cooperative Decision-Making in Merging Area Based on Deep Multi-Agent Reinforcement Learning

Quan Gan, Bin Li (), Zhengang Xiong, Zhenhua Li and Yanyue Liu
Additional contact information
Quan Gan: The State Key Laboratory of Intelligent Transportation System, the Research Institute of Highway Ministry of Transport, Beijing 100088, China
Bin Li: The State Key Laboratory of Intelligent Transportation System, the Highway Monitoring and Emergency Response Center, Ministry of Transport of the China, Beijing 100029, China
Zhengang Xiong: The School of Traffic and Transportation, Beijing Jiaotong University, Beijing 100091, China
Zhenhua Li: The State Key Laboratory of Intelligent Transportation System, the Research Institute of Highway Ministry of Transport, Beijing 100088, China
Yanyue Liu: The State Key Laboratory of Intelligent Transportation System, the Research Institute of Highway Ministry of Transport, Beijing 100088, China

Sustainability, 2024, vol. 16, issue 22, 1-16

Abstract: In recent years, reinforcement learning (RL) methods have shown powerful learning capabilities in single-vehicle autonomous driving. However, few studies have focused on multi-vehicle cooperative driving based on RL, particularly in the dynamically changing traffic environments of highway ramp merge zones. In this paper, a multi-agent deep reinforcement learning (MARL) framework for multi-vehicle cooperative decision-making is proposed based on actor–critic, which categorizes vehicles into two groups according to their origins in the merging area. At the same time, the complexity of the network is reduced and the training process of the model is accelerated by utilizing mechanisms such as partial parameter sharing and experience playback. Additionally, a combination of global and individual rewards is adopted to promote cooperation in connected autonomous vehicles (CAVs) and balance individual and group interests. The training performance of the model is compared under three traffic densities, and our method is also compared with state-of-the-art benchmark methods. The simulation results show that the proposed MARL framework can have stronger policy learning capability and stability under various traffic flow conditions. Moreover, it can also effectively improve the speed of vehicles in the merging zone and reduce traffic conflicts.

Keywords: multi-agent deep reinforcement learning; actor–critic; connected autonomous vehicles; ramp merge; mixed rewards (search for similar items in EconPapers)
JEL-codes: O13 Q Q0 Q2 Q3 Q5 Q56 (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2071-1050/16/22/9646/pdf (application/pdf)
https://www.mdpi.com/2071-1050/16/22/9646/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jsusta:v:16:y:2024:i:22:p:9646-:d:1514649

Access Statistics for this article

Sustainability is currently edited by Ms. Alexandra Wu

More articles in Sustainability from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().