Multi-agent deep reinforcement learning for adaptive coordinated metro service operations with flexible train composition

Cheng-shuo Ying, Andy H.F. Chow, Hoa T.M. Nguyen and Kwai-Sang Chin

Transportation Research Part B: Methodological, 2022, vol. 161, issue C, 36-59

Abstract: This paper presents an adaptive control system for coordinated metro operations with flexible train composition by using a multi-agent deep reinforcement learning (MADRL) approach. The control problem is formulated as a Markov decision process (MDP) with multiple agents regulating different service lines in a metro network with passenger transfer. To ensure the overall computational effectiveness and stability of the control system, we adopt an actor–critic reinforcement learning framework in which each control agent is associated with a critic function for estimating future system states and an actor function deriving local operational decisions. The critics and actors in the MADRL are represented by multi-layer artificial neural networks (ANNs). A multi-agent deep deterministic policy gradient (MADDPG) algorithm is developed for training the actor and critic ANNs through successive simulated transitions over the entire metro network. The developed framework is tested with a real-world scenario in Bakerloo and Victoria Lines of London Underground, UK. Experiment results demonstrate that the proposed method can outperform previous centralized optimization and distributed control approaches in terms of solution quality and performance achieved. Further analysis shows the merits of MADRL for coordinated service regulation with flexible train composition. This study contributes to real-time coordinated metro network services with flexible train composition and advanced optimization techniques.

Keywords: Metro service coordination; Flexible train composition; Multi-agent deep reinforcement learning; Actor–critic architecture; Markov decision process (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0191261522000728
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:transb:v:161:y:2022:i:c:p:36-59

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/supportfaq.cws_home/regional
https://shop.elsevie ... _01_ooc_1&version=01

DOI: 10.1016/j.trb.2022.05.001

Access Statistics for this article

Transportation Research Part B: Methodological is currently edited by Fred Mannering

More articles in Transportation Research Part B: Methodological from Elsevier
Bibliographic data for series maintained by Catherine Liu ().