Distributed multi-agent reinforcement learning approach for energy-saving optimization under disturbance conditions

Dahan Wang, Jianjun Wu, Ximing Chang and Haodong Yin

Transportation Research Part E: Logistics and Transportation Review, 2025, vol. 200, issue C

Abstract: Urban rail transit systems exhibit substantial energy consumption, underpinning the significance of energy-saving optimization strategies for train timetables. Conventionally, trains operate according to an energy-efficient timetable formulated offline. However, station incidents and disturbances often result in deviations from the planned schedule, leading to additional energy expenditure. To address this challenge, the current study introduces a distributed multi-agent reinforcement learning approach(DMARL) for real-time energy-efficient optimization of train timetables. Initially, trains are conceptualized as agents, adopting the Actor-Critic network structure as the learning paradigm, with a distributed deployment architecture facilitating the training of the model. During the interaction phase between agents and the subway system, a progressive reward mechanism is designed to encourage efficient exploratory actions by the agents. In the final case study, data from Shanghai Metro Line 1(SML1) was utilized to demonstrate the effectiveness of the proposed method. The results indicate that when disturbances occur at stations, necessitating extended stop times, the method presented in this paper exhibited stable performance and faster convergence rates in both two-train and three-train systems. Compared to the energy consumption without any action, the energy savings were enhanced by 14.11 % and 11 %, respectively. The timetable updates were completed within milliseconds, confirming the efficacy of the method and its compliance with real-time updating requirements.

Keywords: Multi-agent reinforcement learning; Train energy conservation; Timetable optimization; Station disturbance scenarios; Actor-critic architecture (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S1366554525002212
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:transe:v:200:y:2025:i:c:s1366554525002212

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/bibliographic
http://www.elsevier. ... 600244/bibliographic

DOI: 10.1016/j.tre.2025.104180

Access Statistics for this article

Transportation Research Part E: Logistics and Transportation Review is currently edited by W. Talley

More articles in Transportation Research Part E: Logistics and Transportation Review from Elsevier
Bibliographic data for series maintained by Catherine Liu ().