Economics at your fingertips  

Technical Note—Cyclic Variables and Markov Decision Processes

Avery Haviv ()
Additional contact information
Avery Haviv: Simon Business School, University of Rochester, Rochester, New York 14620

Operations Research, 2020, vol. 68, issue 4, 1231-1237

Abstract: In this paper I develop a cyclic value function iteration, which is an adjustment to the standard value function iteration. When using this algorithm, the inclusion of cyclic variables of any size into the state space of an infinite horizon Markov decision process does not increase the computational complexity of solving for the value function. This result is proven theoretically and shown to closely hold in practice using Monte Carlo simulations.

Keywords: Markov decision process; structural dynamic models; infinite-horizon dynamic programming; econometrics; cyclic variables; seasonality (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed

Downloads: (external link) (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link:

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Matthew Walls ().

Page updated 2020-10-24
Handle: RePEc:inm:oropre:v:68:y:2020:i:4:p:1231-1237