Technical Note—Cyclic Variables and Markov Decision Processes
Avery Haviv ()
Additional contact information
Avery Haviv: Simon Business School, University of Rochester, Rochester, New York 14620
Operations Research, 2020, vol. 68, issue 4, 1231-1237
In this paper I develop a cyclic value function iteration, which is an adjustment to the standard value function iteration. When using this algorithm, the inclusion of cyclic variables of any size into the state space of an infinite horizon Markov decision process does not increase the computational complexity of solving for the value function. This result is proven theoretically and shown to closely hold in practice using Monte Carlo simulations.
Keywords: Markov decision process; structural dynamic models; infinite-horizon dynamic programming; econometrics; cyclic variables; seasonality (search for similar items in EconPapers)
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed
Downloads: (external link)
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:68:y:2020:i:4:p:1231-1237
Access Statistics for this article
More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Matthew Walls ().