UAV Path Planning Based on Random Obstacle Training and Linear Soft Update of DRL in Dense Urban Environment

Yanfei Zhu, Yingjie Tan, Yongfa Chen, Liudan Chen and Kwang Y. Lee ()
Additional contact information
Yanfei Zhu: School of Automation, Guangdong University of Technology, Guangzhou 510006, China
Yingjie Tan: School of Automation, Guangdong University of Technology, Guangzhou 510006, China
Yongfa Chen: School of Automation, Guangdong University of Technology, Guangzhou 510006, China
Liudan Chen: School of Automation, Guangdong University of Technology, Guangzhou 510006, China
Kwang Y. Lee: Department of Electrical and Computer Engineering, Baylor University, Waco, TX 76798, USA

Energies, 2024, vol. 17, issue 11, 1-19

Abstract: The three-dimensional (3D) path planning problem of an Unmanned Aerial Vehicle (UAV) considering the effect of environmental wind in a dense city is investigated in this paper. The mission of the UAV is to fly from its initial position to its destination while ensuring safe flight. The dense obstacle avoidance and the energy consumption in 3D space need to be considered during the mission, which are often ignored in common studies. To solve these problems, an improved Deep Reinforcement Learning (DRL) path planning algorithm based on Double Deep Q-Network (DDQN) is proposed in this paper. Among the algorithms, the random obstacle training method is first proposed to make the algorithm consider various flight scenarios more globally and comprehensively and improve the algorithm’s robustness and adaptability. Then, the linear soft update strategy is employed to realize the smooth neural network parameter update, which enhances the stability and convergence of the training. In addition, the wind disturbances are integrated into the energy consumption model and reward function, which can effectively describe the wind disturbances during the UAV mission to achieve the minimum drag flight. To prevent the neural network from interfering with training failures, the meritocracy mechanism is proposed to enhance the algorithm’s stability. The effectiveness and applicability of the proposed method are verified through simulation analysis and comparative studies. The UAV based on this algorithm has good autonomy and adaptability, which provides a new way to solve the UAV path planning problem in dense urban scenes.

Keywords: unmanned aerial vehicle (UAV); path planning; obstacle avoidance; deep reinforcement learning; double deep Q network (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/17/11/2762/pdf (application/pdf)
https://www.mdpi.com/1996-1073/17/11/2762/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:17:y:2024:i:11:p:2762-:d:1409133

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().