An Intelligent TCP Congestion Control Method Based on Deep Q Network

Yinfeng Wang, Longxiang Wang and Xiaoshe Dong
Additional contact information
Yinfeng Wang: College of Software, Shenzhen Institute of Information Technology, Shenzhen 518116, China
Longxiang Wang: College of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China
Xiaoshe Dong: College of Computer Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China

Future Internet, 2021, vol. 13, issue 10, 1-14

Abstract: To optimize the data migration performance between different supercomputing centers in China, we present TCP-DQN, which is an intelligent TCP congestion control method based on DQN (Deep Q network). The TCP congestion control process is abstracted as a partially observed Markov decision process. In this process, an agent is constructed to interact with the network environment. The agent adjusts the size of the congestion window by observing the characteristics of the network state. The network environment feeds back the reward to the agent, and the agent tries to maximize the expected reward in an episode. We designed a weighted reward function to balance the throughput and delay. Compared with traditional Q-learning, DQN uses double-layer neural networks and experience replay to reduce the oscillation problem that may occur in gradient descent. We implemented the TCP-DQN method and compared it with mainstream congestion control algorithms such as cubic, Highspeed and NewReno. The results show that the throughput of TCP-DQN can reach more than 2 times of the comparison method while the latency is close to the three compared methods.

Keywords: congestion control; reinforcement learning; TCP (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/1999-5903/13/10/261/pdf (application/pdf)
https://www.mdpi.com/1999-5903/13/10/261/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:13:y:2021:i:10:p:261-:d:652723

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().