A Dynamic Holding Approach to Stabilizing a Bus Line Based on the Q-Learning Algorithm with Multistage Look-Ahead

Sheng-Xue He (), Jian-Jia He (), Shi-Dong Liang (), June Qiong Dong () and Peng-Cheng Yuan ()
Additional contact information
Sheng-Xue He: Business School, University of Shanghai for Science and Technology, Shanghai 200093, China
Jian-Jia He: Business School, University of Shanghai for Science and Technology, Shanghai 200093, China
Shi-Dong Liang: Business School, University of Shanghai for Science and Technology, Shanghai 200093, China
June Qiong Dong: School of Business, State University of New York at Oswego, Oswego, New York, 13126
Peng-Cheng Yuan: Business School, University of Shanghai for Science and Technology, Shanghai 200093, China

Transportation Science, 2022, vol. 56, issue 1, 31-51

Abstract: The unreliable service and the unstable operation of a high-frequency bus line are shown as bus bunching and the uneven distribution of headways along the bus line. Although many control strategies, such as the static and dynamic holding strategies, have been proposed to solve the above problems, many of them take on some oversimplified assumptions about the real bus line operation. So it is hard for them to continuously adapt to the evolving complex system. In view of this dynamic setting, we present an adaptive holding method that combines the classic approximate dynamic programming (ADP) with the multistage look-ahead mechanism. The holding time, the only control means used in this study, will be determined by estimating its impact on the operation stability of the bus line system in the remaining observation period. The multistage look-ahead mechanism introduced into the classic Q-learning algorithm of the ADP model makes it easy that the algorithm gets through its earlier unstable phase more quickly and easily. During the implementation of the new holding approach, the past experiences of holding operations can be cumulated effectively into an artificial neural network used to approximate the unavailable Q-factor. The use of a detailed simulation system in the new approach makes it possible to take into account most of the possible causes of instability. The numerical experiments show that the new holding approach can stabilize the system by producing evenly distributed headway and removing bus bunching thoroughly. Compared with the terminal station holding strategies, the new method brings a more reliable bus line with shorter waiting times for passengers.

Keywords: bus bunching; adaptive control; Q-learning; holding; approximate dynamic programming (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/trsc.2021.1048 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ortrsc:v:56:y:2022:i:1:p:31-51

Access Statistics for this article

More articles in Transportation Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().