Soft Actor-Critic and Risk Assessment-Based Reinforcement Learning Method for Ship Path Planning

Jue Wang, Bin Ji () and Qian Fu
Additional contact information
Jue Wang: School of Traffic & Transportation Engineering, Central South University, Changsha 410075, China
Bin Ji: School of Traffic & Transportation Engineering, Central South University, Changsha 410075, China
Qian Fu: School of Traffic & Transportation Engineering, Central South University, Changsha 410075, China

Sustainability, 2024, vol. 16, issue 8, 1-16

Abstract: Ship path planning is one of the most important themes in waterway transportation, which is deemed as the cleanest mode of transportation due to its environmentally friendly and energy-efficient nature. A path-planning method that combines the soft actor-critic (SAC) and navigation risk assessment is proposed to address ship path planning in complex water environments. Specifically, a continuous environment model is established based on the Markov decision process (MDP), which considers the characteristics of the ship path-planning problem. To enhance the algorithm’s performance, an information detection strategy for restricted navigation areas is employed to improve state space, converting absolute bearing into relative bearing. Additionally, a risk penalty based on the navigation risk assessment model is introduced to ensure path safety while imposing potential energy rewards regarding navigation distance and turning angle. Finally, experimental results obtained from a navigation simulation environment verify the robustness of the proposed method. The results also demonstrate that the proposed algorithm achieves a smaller path length and sum of turning angles with safety and fuel economy improvement compared with traditional methods such as RRT (rapidly exploring random tree) and DQN (deep Q-network).

Keywords: ship path planning; navigation efficiency; maximum entropy deep reinforcement learning; soft actor-critic; risk assessment (search for similar items in EconPapers)
JEL-codes: O13 Q Q0 Q2 Q3 Q5 Q56 (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2071-1050/16/8/3239/pdf (application/pdf)
https://www.mdpi.com/2071-1050/16/8/3239/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jsusta:v:16:y:2024:i:8:p:3239-:d:1374794

Access Statistics for this article

Sustainability is currently edited by Ms. Alexandra Wu

More articles in Sustainability from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().