Heterogeneity, reinforcement learning, and chaos in population games

Jakub Bielawski, Thiparat Chotibut (), Fryderyk Falniowski, Michał Misiurewicz and Georgios Piliouras ()
Additional contact information
Jakub Bielawski: a Department of Mathematics , Krakow University of Economics , Kraków 31-510 , Poland
Thiparat Chotibut: b Chula Intelligent and Complex Systems, Department of Physics, Faculty of Science , Chulalongkorn University , Bangkok 10330 , Thailand
Fryderyk Falniowski: a Department of Mathematics , Krakow University of Economics , Kraków 31-510 , Poland
Michał Misiurewicz: c Department of Mathematical Sciences , Indiana University-Purdue University Indianapolis , Indianapolis , IN 46202
Georgios Piliouras: d Google DeepMind , London EC4A 3TW , United Kingdom

Proceedings of the National Academy of Sciences, 2025, vol. 122, issue 25, e2319929121

Abstract:

Inspired by the challenges at the intersection of Evolutionary Game Theory and Machine Learning, we investigate a class of discrete-time multiagent reinforcement learning (MARL) dynamics in population/nonatomic congestion games, where agents have diverse beliefs and learn at different rates. These congestion games, a well-studied class of potential games, are characterized by individual agents having negligible effects on system performance, strongly aligned incentives, and well-understood advantageous properties of Nash equilibria. Despite the presence of static Nash equilibria, we demonstrate that MARL dynamics with heterogeneous learning rates can deviate from these equilibria, exhibiting instability and even chaotic behavior and resulting in increased social costs. Remarkably, even within these chaotic regimes, we show that the time-averaged macroscopic behavior converges to exact Nash equilibria, thus linking the microscopic dynamic complexity with traditional equilibrium concepts. By employing dynamical systems techniques, we analyze the interaction between individual-level adaptation and population-level outcomes, paving the way for studying heterogeneous learning dynamics in discrete time across more complex game scenarios.

Keywords: game theory; evolutionary dynamics; multiplicative weights update; Li-Yorke chaos; congestion games (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1073/pnas.2319929121 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nas:journl:v:122:y:2025:p:e2319929121

Access Statistics for this article

More articles in Proceedings of the National Academy of Sciences from Proceedings of the National Academy of Sciences
Bibliographic data for series maintained by PNAS Product Team ().