Attainability of Boundary Points under Reinforcement Learning

Ed Hopkins and Martin Posch

Edinburgh School of Economics Discussion Paper Series from Edinburgh School of Economics, University of Edinburgh

Abstract: This paper investigates the properties of the most common form of reinforcement learning (the "basic model" of Erev and Roth, American Economic Review, 88, 848-881, 1998). Stochastic approximation theory has been used to analyse the local stability of fixed points under this learning process. However, as we show, when such points are on the boundary of the state space, for example, pure strategy equilibria, standard results from the theory of stochastic approximation do not apply. We offer what we believe to be the correct treatment of boundary points, and provide a new and more general result: this model of learning converges with zero probability to fixed points which are unstable under the Maynard Smith or adjusted version of the evolutionary replicator dynamics. For two player games these are the fixed points that are linearly unstable under the standard replicator dynamics.

Keywords: learning in games; reinforcement learning; stochastic approximation; replicator dynamics (search for similar items in EconPapers)
JEL-codes: C72 C73 D83 (search for similar items in EconPapers)
Pages: 17
Date: 2003-07
New Economics Papers: this item is included in nep-dev
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.econ.ed.ac.uk/papers/id79_esedps.pdf

Related works:
Journal Article: Attainability of boundary points under reinforcement learning (2005)
Working Paper: Attainability of Boundary Points under Reinforcement Learning (2003)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:edn:esedps:79

Access Statistics for this paper

More papers in Edinburgh School of Economics Discussion Paper Series from Edinburgh School of Economics, University of Edinburgh 31 Buccleuch Place, EH8 9JT, Edinburgh. Contact information at EDIRC.
Bibliographic data for series maintained by Research Office ().