Predictive reinforcement learning: map-less navigation method for mobile robot

Dmitrii Dobriborsci (), Roman Zashchitin (), Mikhail Kakanov (), Wolfgang Aumer () and Pavel Osinenko ()
Additional contact information
Dmitrii Dobriborsci: Deggendorf Institute of Technology
Roman Zashchitin: Deggendorf Institute of Technology
Mikhail Kakanov: Deggendorf Institute of Technology
Wolfgang Aumer: Deggendorf Institute of Technology
Pavel Osinenko: Skolkovo Institute of Technology

Journal of Intelligent Manufacturing, 2024, vol. 35, issue 8, No 31, 4217-4232

Abstract: Abstract The application of reinforcement learning in mobile robotics faces the challenges of real-world physical environments, in contrast to playground setups like video games. In a mobile robot motion control, it is not always possible to perform episodes of pre-training in large amounts due to time, resource limitations or other concerns. Control methods that rely on a prior explicit map may be impractical or even impossible to use for new dynamic environments. In this paper, we present a method of local navigation approach for driving a robot to a desired position without relying on an explicit map of the environment. Only the laser scan measurements were used to determine the obstacles. We focus in this work on online methods of reinforcement learning which do not require running the robot in full episodes until success or failure. However, the price for such an online capability is that some model knowledge about the environment has to be utilized. Here, we propose an algorithm called stacked Q-learning, which unifies aspects of standard reinforcement learning techniques with model-based predictive agents. We compare this algorithm to a classical model predictive controller. The comparison focuses on the accumulated cost of parking the robot avoiding obstacles. The results look promising as the stacked Q-learning beat its counterpart, model predictive control, yet being of the same computational complexity. The suggested agent design of stacked Q-learning can thus be taken as a foundation for a class of predictive reinforcement learning methods.

Keywords: Reinforcement learning; Mobile robotics; Predictive control; Mapless navigation (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10845-023-02197-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joinma:v:35:y:2024:i:8:d:10.1007_s10845-023-02197-y

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10845

DOI: 10.1007/s10845-023-02197-y

Access Statistics for this article

Journal of Intelligent Manufacturing is currently edited by Andrew Kusiak

More articles in Journal of Intelligent Manufacturing from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().