A Hybrid Multi-Agent Reinforcement Learning Approach for Spectrum Sharing in Vehicular Networks

Mansoor Jamal, Zaib Ullah (), Muddasar Naeem, Musarat Abbas and Antonio Coronato
Additional contact information
Mansoor Jamal: Department of Electronics, Quaid-i-Azam University, Islamabad 44000, Pakistan
Zaib Ullah: Artificial Intelligence and Robotics Lab, Università Telematica Giustino Fortunato, 82100 Benevento, Italy
Muddasar Naeem: Artificial Intelligence and Robotics Lab, Università Telematica Giustino Fortunato, 82100 Benevento, Italy
Musarat Abbas: Department of Electronics, Quaid-i-Azam University, Islamabad 44000, Pakistan
Antonio Coronato: Artificial Intelligence and Robotics Lab, Università Telematica Giustino Fortunato, 82100 Benevento, Italy

Future Internet, 2024, vol. 16, issue 5, 1-15

Abstract: Efficient spectrum sharing is essential for maximizing data communication performance in Vehicular Networks (VNs). In this article, we propose a novel hybrid framework that leverages Multi-Agent Reinforcement Learning (MARL), thereby combining both centralized and decentralized learning approaches. This framework addresses scenarios where multiple vehicle-to-vehicle (V2V) links reuse the frequency spectrum preoccupied by vehicle-to-infrastructure (V2I) links. We introduce the QMIX technique with the Deep Q Networks (DQNs) algorithm to facilitate collaborative learning and efficient spectrum management. The DQN technique uses a neural network to approximate the Q value function in high-dimensional state spaces, thus mapping input states to (action, Q value) tables that facilitate self-learning across diverse scenarios. Similarly, the QMIX is a value-based technique for multi-agent environments. In the proposed model, each V2V agent having its own DQN observes the environment, receives observation, and obtains a common reward. The QMIX network receives Q values from all agents considering individual benefits and collective objectives. This mechanism leads to collective learning while V2V agents dynamically adapt to real-time conditions, thus improving VNs performance. Our research finding highlights the potential of hybrid MARL models for dynamic spectrum sharing in VNs and paves the way for advanced cooperative learning strategies in vehicular communication environments. Furthermore, we conducted an in-depth exploration of the simulation environment and performance evaluation criteria, thus concluding in a comprehensive comparative analysis with cutting-edge solutions in the field. Simulation results show that the proposed framework efficiently performs against the benchmark architecture in terms of V2V transmission probability and V2I peak data transfer.

Keywords: vehicular networks; multi-agent reinforcement learning; deep Q-networks (DQN); QMIX; spectrum management; deep reinforcement learning (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/16/5/152/pdf (application/pdf)
https://www.mdpi.com/1999-5903/16/5/152/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:16:y:2024:i:5:p:152-:d:1384960

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().