Models and algorithms for skip-free Markov decision processes on trees

Edmund J Collins
Additional contact information
Edmund J Collins: University of Bristol, Bristol, UK

Journal of the Operational Research Society, 2015, vol. 66, issue 10, 1595-1604

Abstract: We introduce a class of models for multidimensional control problems that we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in the negative direction. Starting with the finite average cost case, we show that the algorithm combines the advantages of both value iteration and policy iteration—it is guaranteed to converge to an optimal policy and optimal value function after a finite number of iterations but the computational effort required for each iteration step is comparable with that for value iteration. We show that the algorithm can also be used to solve discounted cost models and continuous-time models, and that a suitably modified algorithm can be used to solve communicating models.

Date: 2015
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.palgrave-journals.com/jors/journal/v66/n10/pdf/jors201463a.pdf Link to full text PDF (application/pdf)
http://www.palgrave-journals.com/jors/journal/v66/n10/full/jors201463a.html Link to full text HTML (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pal:jorsoc:v:66:y:2015:i:10:p:1595-1604

Ordering information: This journal article can be ordered from
http://www.springer. ... search/journal/41274

Access Statistics for this article

Journal of the Operational Research Society is currently edited by Tom Archibald and Jonathan Crook

More articles in Journal of the Operational Research Society from Palgrave Macmillan, The OR Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().