Percentile Optimization for Markov Decision Processes with Parameter Uncertainty

Erick Delage () and Shie Mannor ()
Additional contact information
Erick Delage: Department of Management Science, HEC Montréal, Montréal, Quebec H3T 2A7, Canada
Shie Mannor: Department of Electrical and Computer Engineering, McGill University, Montreal, Quebec H3A 2A7, Canada

Operations Research, 2010, vol. 58, issue 1, 203-213

Abstract: Markov decision processes are an effective tool in modeling decision making in uncertain dynamic environments. Because the parameters of these models typically are estimated from data or learned from experience, it is not surprising that the actual performance of a chosen strategy often differs significantly from the designer's initial expectations due to unavoidable modeling ambiguity. In this paper, we present a set of percentile criteria that are conceptually natural and representative of the trade-off between optimistic and pessimistic views of the question. We study the use of these criteria under different forms of uncertainty for both the rewards and the transitions. Some forms are shown to be efficiently solvable and others highly intractable. In each case, we outline solution concepts that take parametric uncertainty into account in the process of decision making.

Keywords: Markov decision processes; parameter uncertainty; finite state; stochastic model applications; stochastic programming; value at risk; chance-constrained optimization (search for similar items in EconPapers)
Date: 2010
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (23)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.1080.0685 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:58:y:2010:i:1:p:203-213

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().