Markovian Decision Processes with Uncertain Transition Probabilities

Jay K. Satia and Roy E. Lave
Additional contact information
Jay K. Satia: Northeastern University, Bostom, Massachusetts
Roy E. Lave: Stanford University, Stanford, California

Operations Research, 1973, vol. 21, issue 3, 728-740

Abstract: This paper examines Markovian decision processes in which the transition probabilities corresponding to alternative decisions are not known with certainty. The processes are assumed to be finite-state, discrete-time, and stationary. The rewards axe time discounted. Both a game-theoretic and the Bayesian formulation are considered. In the game-theoretic formulation, variants of a policy-iteration algorithm are provided for both the max-min and the max-max cases. An implicit enumeration algorithm is discussed for the Bayesian formulation where upper and lower bounds on the total expected discounted return are provided by the max-max and max-min optimal policies. Finally, the paper discusses asymptotically Bayes-optimal policies.

Date: 1973
References: Add references at CitEc
Citations: View citations in EconPapers (32)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.21.3.728 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:21:y:1973:i:3:p:728-740

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().