Applications of Markov Chain Approximation Methods to Optimal Control Problems in Economics

Keyvan Eslami and Thomas Phelan

No 21-04R, Working Papers from Federal Reserve Bank of Cleveland

Abstract: In this paper we explore some benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems in economics. We first show that the implicit finite-difference scheme of Achdou et al. (2022) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate that, relative to the implicit finite-difference approach, using variations of modified policy function iteration to solve income fluctuation problems both with and without discrete choices can lead to an increase in the speed of convergence of more than an order of magnitude. Finally, we provide several consistent chain constructions for stationary portfolio problems with correlated state variables, and illustrate the flexibility of the MCA approach by using it to construct and compare two simple solution methods for a general equilibrium model with financial frictions.

Keywords: Dynamic programming; financial frictions; Numerical methods; Markov chain approximation (search for similar items in EconPapers)
JEL-codes: C63 E00 G11 (search for similar items in EconPapers)
Pages: 50
Date: 2021-02-10, Revised 2022-05-17
New Economics Papers: this item is included in nep-cwa, nep-mac and nep-ore
Note: Replication materials may be found at https://github.com/tphelanECON/EslamiPhelan_MCA
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://doi.org/10.26509/frbc-wp-202104r Full Text (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:fip:fedcwq:89860

Ordering information: This working paper can be ordered from

DOI: 10.26509/frbc-wp-202104r

Access Statistics for this paper

More papers in Working Papers from Federal Reserve Bank of Cleveland Contact information at EDIRC.
Bibliographic data for series maintained by 4D Library ().