Computational Performance of Deep Reinforcement Learning to Find Nash Equilibria

Christoph Graf (), Viktor Zobernig (), Johannes Schmidt () and Claude Klöckl ()
Additional contact information
Christoph Graf: New York University
Viktor Zobernig: University of Natural Resources and Life Sciences
Johannes Schmidt: University of Natural Resources and Life Sciences
Claude Klöckl: University of Natural Resources and Life Sciences

Computational Economics, 2024, vol. 63, issue 2, No 4, 529-576

Abstract: Abstract We test the performance of deep deterministic policy gradient—a deep reinforcement learning algorithm, able to handle continuous state and action spaces—to find Nash equilibria in a setting where firms compete in offer prices through a uniform price auction. These algorithms are typically considered “model-free” although a large set of parameters is utilized by the algorithm. These parameters may include learning rates, memory buffers, state space dimensioning, normalizations, or noise decay rates, and the purpose of this work is to systematically test the effect of these parameter configurations on convergence to the analytically derived Bertrand equilibrium. We find parameter choices that can reach convergence rates of up to 99%. We show that the algorithm also converges in more complex settings with multiple players and different cost structures. Its reliable convergence may make the method a useful tool to studying strategic behavior of firms even in more complex settings.

Keywords: Bertrand equilibrium; Competition in uniform price auctions; Deep deterministic policy gradient algorithm; DDPG; Parameter sensitivity analysis (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10614-022-10351-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:kap:compec:v:63:y:2024:i:2:d:10.1007_s10614-022-10351-6

Ordering information: This journal article can be ordered from
http://www.springer. ... ry/journal/10614/PS2

DOI: 10.1007/s10614-022-10351-6

Access Statistics for this article

Computational Economics is currently edited by Hans Amman

More articles in Computational Economics from Springer, Society for Computational Economics Contact information at EDIRC.
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().