Scalable Reinforcement Learning for Multiagent Networked Systems

Guannan Qu (), Adam Wierman () and Na Li ()
Additional contact information
Guannan Qu: Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213
Adam Wierman: Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, California 91125
Na Li: School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138

Operations Research, 2022, vol. 70, issue 6, 3601-3628

Abstract: We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized. A fundamental challenge in this setting is that the state-action space size scales exponentially in the number of agents, rendering the problem intractable for large networks. In this paper, we propose a scalable actor critic (SAC) framework that exploits the network structure and finds a localized policy that is an O ( ρ κ + 1 ) -approximation of a stationary point of the objective for some ρ ∈ ( 0 , 1 ) , with complexity that scales with the local state-action space size of the largest κ -hop neighborhood of the network. We illustrate our model and approach using examples from wireless communication, epidemics, and traffic.

Keywords: Stochastic Models; stochastic systems; networked systems; reinforcement learning (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2021.2226 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:70:y:2022:i:6:p:3601-3628

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().