Multiagent Reinforcement Learning Based on Fusion-Multiactor-Attention-Critic for Multiple-Unmanned-Aerial-Vehicle Navigation Control

Sangwoo Jeon, Hoeun Lee, Vishnu Kumar Kaliappan (), Tuan Anh Nguyen, Hyungeun Jo, Hyeonseo Cho and Dugki Min ()
Additional contact information
Sangwoo Jeon: Department of Computer Science and Engineering, Konkuk University, Seoul 05029, Korea
Hoeun Lee: Department of Computer Science and Engineering, Konkuk University, Seoul 05029, Korea
Vishnu Kumar Kaliappan: Konkuk Aerospace Design-Airworthiness Research Institute, Konkuk University, Seoul 05029, Korea
Tuan Anh Nguyen: Konkuk Aerospace Design-Airworthiness Research Institute, Konkuk University, Seoul 05029, Korea
Hyungeun Jo: Department of Computer Science and Engineering, Konkuk University, Seoul 05029, Korea
Hyeonseo Cho: Department of Computer Science and Engineering, Konkuk University, Seoul 05029, Korea
Dugki Min: Department of Computer Science and Engineering, Konkuk University, Seoul 05029, Korea

Energies, 2022, vol. 15, issue 19, 1-18

Abstract: The proliferation of unmanned aerial vehicles (UAVs) has spawned a variety of intelligent services, where efficient coordination plays a significant role in increasing the effectiveness of cooperative execution. However, due to the limited operational time and range of UAVs, achieving highly efficient coordinated actions is difficult, particularly in unknown dynamic environments. This paper proposes a multiagent deep reinforcement learning (MADRL)-based fusion-multiactor-attention-critic (F-MAAC) model for multiple UAVs’ energy-efficient cooperative navigation control. The proposed model is built on the multiactor-attention-critic (MAAC) model, which offers two significant advances. The first is the sensor fusion layer, which enables the actor network to utilize all required sensor information effectively. Next, a layer that computes the dissimilarity weights of different agents is added to compensate for the information lost through the attention layer of the MAAC model. We utilize the UAV LDS (logistic delivery service) environment created by the Unity engine to train the proposed model and verify its energy efficiency. The feature that measures the total distance traveled by the UAVs is incorporated with the UAV LDS environment to validate the energy efficiency. To demonstrate the performance of the proposed model, the F-MAAC model is compared with several conventional reinforcement learning models with two use cases. First, we compare the F-MAAC model to the DDPG, MADDPG, and MAAC models based on the mean episode rewards for 20k episodes of training. The two top-performing models (F-MAAC and MAAC) are then chosen and retrained for 150k episodes. Our study determines the total amount of deliveries done within the same period and the total amount done within the same distance to represent energy efficiency. According to our simulation results, the F-MAAC model outperforms the MAAC model, making 38% more deliveries in 3000 time steps and 30% more deliveries per 1000 m of distance traveled.

Keywords: air logistics; multiagent reinforcement learning; actor-attention-critic; sensor fusion; multiple UAV (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/15/19/7426/pdf (application/pdf)
https://www.mdpi.com/1996-1073/15/19/7426/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:15:y:2022:i:19:p:7426-:d:937788

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().