An Energy-Efficient Cross-Layer Routing Protocol for Cognitive Radio Networks Using Apprenticeship Deep Reinforcement Learning

Yihang Du, Ying Xu, Lei Xue, Lijia Wang and Fan Zhang
Additional contact information
Yihang Du: Electronic Countermeasure Institute, National University of Defense Technology, Shushan District, Hefei 230000, China
Ying Xu: Electronic Countermeasure Institute, National University of Defense Technology, Shushan District, Hefei 230000, China
Lei Xue: Electronic Countermeasure Institute, National University of Defense Technology, Shushan District, Hefei 230000, China
Lijia Wang: CTTL-Terminals of China Academy of Telecommunication Research of MIIT, Haidian District, Beijing 100191, China
Fan Zhang: Science and Technology Research Bureau of AnHui XinHua University, Shushan District, Hefei 230000, China

Energies, 2019, vol. 12, issue 14, 1-21

Abstract: Deep reinforcement learning (DRL) has been successfully used for the joint routing and resource management in large-scale cognitive radio networks. However, it needs lots of interactions with the environment through trial and error, which results in large energy consumption and transmission delay. In this paper, an apprenticeship learning scheme is proposed for the energy-efficient cross-layer routing design. Firstly, to guarantee energy efficiency and compress huge action space, a novel concept called dynamic adjustment rating is introduced, which regulates transmit power efficiently with multi-level transition mechanism. On top of this, the Prioritized Memories Deep Q-learning from Demonstrations (PM-DQfD) is presented to speed up the convergence and reduce the memory occupation. Then the PM-DQfD is applied to the cross-layer routing design for power efficiency improvement and routing latency reduction. Simulation results confirm that the proposed method achieves higher energy efficiency, shorter routing latency and larger packet delivery ratio compared to traditional algorithms such as Cognitive Radio Q-routing (CRQ-routing), Prioritized Memories Deep Q-Network (PM-DQN), and Conjecture Based Multi-agent Q-learning Scheme (CBMQ).

Keywords: cognitive radio networks; energy efficiency; apprenticeship learning; dynamic adjustment rating; Prioritized Memories deep Q-learning from Demonstrations (search for similar items in EconPapers)
JEL-codes: Q Q0 Q4 Q40 Q41 Q42 Q43 Q47 Q48 Q49 (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1996-1073/12/14/2829/pdf (application/pdf)
https://www.mdpi.com/1996-1073/12/14/2829/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jeners:v:12:y:2019:i:14:p:2829-:d:250678

Access Statistics for this article

Energies is currently edited by Ms. Agatha Cao

More articles in Energies from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().