Cooperative traffic optimization with multi-agent reinforcement learning and evolutionary strategy: Bridging the gap between micro and macro traffic control

Jianshuai Feng, Kaize Lin, Tianyu Shi, Yuankai Wu, Yong Wang, Hailong Zhang and Huachun Tan

Physica A: Statistical Mechanics and its Applications, 2024, vol. 647, issue C

Abstract: The emergence of connected and autonomous vehicles (CAVs) holds promise for fine-grained traffic control. However, due to the longevity of future mixed traffic scenarios, there is a need for an in-depth exploration of integrating the microscopic speed control of CAVs with the macroscopic variable speed limit (VSL) of human-driven vehicles (HDVs). This paper proposes a Cooperative Traffic Optimization with Multi-agent Reinforcement Learning and Evolutionary VSL (CTO-ME) framework, which combines microscopic CAV control with macroscopic VSL control. The framework incorporates a Graph Attention Mechanism (GATs) into the multi-agent reinforcement learning framework for intelligent decision-making by microscopic-level vehicles. Additionally, an evolutionary strategy is developed to design the VSL network architecture, enabling macroscopic level real-time speed limit adjustments based on infrastructure. A multi-objective reward function is proposed to optimize both micro and macro efficiency and safety, accounting for both vehicle behavior and traffic flow. Experiments on the designed Bottleneck traffic scenarios show that the proposed approach, CTO-ME, is able to achieve superior performance and outperforms other baselines in terms of traffic throughput, average speed, and safety. Specifically, CTO-ME enhances average velocity by 37%, increases overall throughput by 309%, and raises arrival ratio by 70% than traditional Intelligent Driver Model (IDM).

Keywords: Variable speed limit; Connected and automated vehicles; Multi-agent reinforcement learning; Evolution strategy; Graph neural network (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378437124002437
Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:phsmap:v:647:y:2024:i:c:s0378437124002437

DOI: 10.1016/j.physa.2024.129734

Access Statistics for this article

Physica A: Statistical Mechanics and its Applications is currently edited by K. A. Dawson, J. O. Indekeu, H.E. Stanley and C. Tsallis

More articles in Physica A: Statistical Mechanics and its Applications from Elsevier
Bibliographic data for series maintained by Catherine Liu ().