A Two-State Partially Observable Markov Decision Process with Uniformly Distributed Observations

Abraham Grosfeld-Nir
Additional contact information
Abraham Grosfeld-Nir: Tel Aviv University, Tel Aviv, Israel

Operations Research, 1996, vol. 44, issue 3, 458-463

Abstract: A controller observes a production system periodically, over time. If the system is in the GOOD state during one period, there is a constant probability that it will deteriorate and be in the BAD state during the next period (and remains there). The true state of the system is unobservable and can only be inferred from observations (quality of output). Two actions are available: CONTINUE or REPLACE (for a fixed cost). The objective is to maximize the expected discounted value of the total future income. For both the finite- and infinite-horizon problems, the optimal policy is of a CONTROL LIMIT (CLT) type: continue if the good state probability exceeds the CLT, and replace otherwise. The computation of the CLT involves a functional equation. An analytical solution for this equation is as yet unknown.For uniformly distributed observations we obtain the infinite-horizon CLT analytically . We also show that the finite horizon CLTs, as a function of the time remaining, are not necessarily monotone, which is counterintuitive.

Keywords: dynamic programming/optimal control; Markov; finite state; inventory/production; maintenance/replacement (search for similar items in EconPapers)
Date: 1996
References: Add references at CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.44.3.458 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:44:y:1996:i:3:p:458-463

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().