Anisotropic Q-learning and waiting estimation based real-time routing for automated guided vehicles at container terminals

Pengfei Zhou (), Li Lin and Kap Hwan Kim
Additional contact information
Pengfei Zhou: Dalian University of Technology
Li Lin: Dalian Jiaotong University
Kap Hwan Kim: Pusan National University

Journal of Heuristics, 2023, vol. 29, issue 2, No 1, 207-228

Abstract: Abstract Finding short and convenient routes for vehicles is an important issue on efficient operations of Automated Guided Vehicle (AGV) systems at container terminals. This paper proposes an anisotropic Q-learning method for AGVs to find the shortest-time routes in the guide-path network of cross-lane type according to real-time vehicle states, which includes current and destination positions, heading direction and the number of vehicles at anisotropic four-direction neighboring locations. The vehicle waiting time of AGV systems is discussed and its estimation is suggested to improve the policy to select actions in the Q-learning method. An improved anisotropic Q-learning routing algorithm is developed with the vehicle-waiting-time-estimation based selecting-action policy. The parameter settings and performance of the proposed methods are analyzed based on simulations. The numerical experiments show that the improved anisotropic Q-learning method can provide stable and dynamic solutions for AGV routing, and achieve 9.5% improvement in optimization efficiency compared to the Jeon Learning Method (Jeon et al. in Logist Res 3(1):19–27, 2011).

Keywords: Container transportation; Automated guided vehicle routing; Q-learning algorithm; Waiting time estimation (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10732-020-09463-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joheur:v:29:y:2023:i:2:d:10.1007_s10732-020-09463-9

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10732

DOI: 10.1007/s10732-020-09463-9

Access Statistics for this article

Journal of Heuristics is currently edited by Manuel Laguna

More articles in Journal of Heuristics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().