Structural Reinforcement Learning for Heterogeneous Agent Macroeconomics

Yucheng Yang, Chiyuan Wang, Andreas Schaab and Benjamin Moll

No 20980, CEPR Discussion Papers from Centre for Economic Policy Research

Abstract: We present a new approach to formulating and solving heterogeneous agent models with aggregate risk. We replace the cross-sectional distribution with low-dimensional prices as state variables and let agents learn equilibrium price dynamics directly from simulated paths. To do so, we introduce a "structural reinforcement learning" (SRL) method which treats prices via simulation while exploiting agents’ structural knowledge of their own individual dynamics. Our SRL method yields a general and highly efficient global solution method for heterogeneous agent models that sidesteps the Master equation and handles models traditional methods struggle with, like those with nontrivial market-clearing conditions. We illustrate the approach in the Krusell-Smith model, the Huggett model with aggregate shocks, and a HANK model with a forward-looking Phillips curve, all of which we solve globally within minutes.

Keywords: Reinforcement; learning (search for similar items in EconPapers)
JEL-codes: E00 (search for similar items in EconPapers)
Date: 2025-12
References: Add references at CitEc
Citations:

Downloads: (external link)
https://cepr.org/publications/DP20980 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cpr:ceprdp:20980

Ordering information: This working paper can be ordered from
https://cepr.org/publications/DP20980

Access Statistics for this paper

More papers in CEPR Discussion Papers from Centre for Economic Policy Research 33 Great Sutton Street, London EC1V 0DX, UK.
Bibliographic data for series maintained by CEPR ().