A Novel Model for Optimizing Roundabout Merging Decisions Based on Markov Decision Process and Force-Based Reward Function

Qingyuan Shen, Haobin Jiang (), Aoxue Li () and Shidian Ma
Additional contact information
Qingyuan Shen: School of Automotive and Traffic Engineering, Jiangsu University, Zhenjiang 212013, China
Haobin Jiang: Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China
Aoxue Li: School of Automotive and Traffic Engineering, Jiangsu University, Zhenjiang 212013, China
Shidian Ma: Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013, China

Mathematics, 2025, vol. 13, issue 6, 1-17

Abstract: Autonomous vehicles (AVs) are increasingly operating in complex traffic environments where safe and efficient decision-making is crucial. Merging into roundabouts is a key interaction scenario. This paper introduces a decision-making approach for roundabout merging that combines human driving behavior with advanced reinforcement learning (RL) techniques to enhance both safety and efficiency. The proposed framework models the decision-making process of AVs at roundabouts as a Markov decision process (MDP), optimizing the state, action, and reward spaces to more accurately reflect real-world driving behaviors. It simplifies the state space using relative distance and speed and defines three action profiles based on real traffic data to replicate human-like driving behavior. A force-based reward function, derived from constitutive relations, simulates vehicle-roundabout interactions, offering detailed, physically consistent feedback that enhances learning results. The results showed that this method effectively replicates human-like driving decisions, supporting the integration of AVs into dynamic traffic environments. Future research should address the challenges related to partial observability and further refine the state, action, and reward spaces. This research lays the groundwork for adaptive and interpretable decision-making frameworks for AVs, contributing to safer and more efficient traffic dynamics at roundabouts.

Keywords: Markov decision process; decision-making; autonomous vehicles; roundabout; merging model (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/6/912/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/6/912/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:6:p:912-:d:1608386

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().