Markov game for CV joint adaptive routing in stochastic traffic networks: A scalable learning approach

Shan Yang and Yang Liu

Transportation Research Part B: Methodological, 2024, vol. 189, issue C

Abstract: This study proposes a learning-based approach to tackle the challenge of joint adaptive routing in stochastic traffic networks with Connected Vehicles (CVs). We introduce a Markov Routing Game (MRG) to model the adaptive routing behavior of all vehicles in such networks, thereby incorporating both competitive route choices and real-time decision-making. We establish the existence of the Nash policy (i.e., optimal joint adaptive routing policy) within the MRG that enables vehicles to adapt optimally to real-time traffic conditions online through efficient communication. To enhance scalability, we innovate with a homogeneity-based mean-field approximation method and, based on that, further develop the Homogeneity-based Mean-Field Deep Reinforcement Learning (HMF-DRL) algorithm to learn the Nash policy within the MRG. Through numerical experiments on the Nguyen–Dupuis network, we demonstrate our algorithm’s ability to efficiently converge and learn the joint adaptive routing policy that significantly enhances traffic network efficiency. Furthermore, our study provides insights into the effects of travel demand, penetration of CVs, and levels of uncertainty on the performance of the joint adaptive routing policy. This paper presents a significant step towards improving network efficiency and reducing the travel time for a majority of vehicles amid uncertain traffic conditions.

Keywords: Markov Routing Game; Connected vehicles; Joint adaptive routing; Mean-field multi-agent reinforcement learning; Stochastic traffic network (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0191261524001218
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:transb:v:189:y:2024:i:c:s0191261524001218

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/supportfaq.cws_home/regional
https://shop.elsevie ... _01_ooc_1&version=01

DOI: 10.1016/j.trb.2024.102997

Access Statistics for this article

Transportation Research Part B: Methodological is currently edited by Fred Mannering

More articles in Transportation Research Part B: Methodological from Elsevier
Bibliographic data for series maintained by Catherine Liu ().