PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES AND PERIODIC POLICIES WITH APPLICATIONS

John Goulionis () and D. Stengos ()
Additional contact information
John Goulionis: Department of Statistics and Insurance Science, University of Piraeus, 80 Karaoli and Dimitriou Street, Piraeus, 18534, Greece
D. Stengos: Department of Statistics and Insurance Science, University of Piraeus, 80 Karaoli and Dimitriou Street, Piraeus, 18534, Greece

International Journal of Information Technology & Decision Making (IJITDM), 2011, vol. 10, issue 06, 1175-1197

Abstract: This paper treats the infinite horizon discounted cost control problem for partially observable Markov decision processes. Sondik studied the class of finitely transient policies and showed that their value functions over an infinite time horizon are piecewise linear (p.w.l) and can be computed exactly by solving a system of linear equations. However, the condition for finite transience is stronger than is needed to ensure p.w.l. value functions. In this paper, we introduce alternatively the class of periodic policies whose value functions turn out to be also p.w.l. Moreover, we examine a more general condition than finite transience and periodicity that ensures p.w.l. value functions. We implement these ideas in a replacement problem under Markovian deterioration, investigate for periodic policies and give numerical examples.

Keywords: Partially observable Markov decision processes; optimization and information technology; information policy making; stochastic models (search for similar items in EconPapers)
Date: 2011
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.worldscientific.com/doi/abs/10.1142/S0219622011004762
Access to full text is restricted to subscribers

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wsi:ijitdm:v:10:y:2011:i:06:n:s0219622011004762

Ordering information: This journal article can be ordered from

DOI: 10.1142/S0219622011004762

Access Statistics for this article

International Journal of Information Technology & Decision Making (IJITDM) is currently edited by Yong Shi

More articles in International Journal of Information Technology & Decision Making (IJITDM) from World Scientific Publishing Co. Pte. Ltd.
Bibliographic data for series maintained by Tai Tone Lim ().