Accounting for parametric uncertainty in Markov decision processes

Adam W. Schapaugh and Andrew J. Tyre

Ecological Modelling, 2013, vol. 254, issue C, 15-21

Abstract: Markov decision processes have become the standard tool for modeling sequential decision-making problems in conservation. In many real-world applications, however, it is practically infeasible to accurately parameterize the state transition function. In this study, we introduce a new way of dealing with ambiguity in the state transition function. In contrast to existing methods, we explore the effects of uncertainty at the level of the policy, rather than at the level of decisions within states. We use information-gap decision theory to ask the question of how much uncertainty in the state transition function can be tolerated while still delivering a specified expected value given by the objective function. Accordingly, the goal of the optimization problem is no longer to maximize expected value, but to maximize local robustness to uncertainty (while still meeting the desired level of performance). We analyze a simple land acquisition problem, using info-gap decision theory to propagate uncertainties and rank alternative policies. Rather than requiring information about the extent of parameter uncertainty at the outset, info-gap addresses the question of how much uncertainty is permissible in the state transition function before the optimal policy would change.

Keywords: Information-gap; Markov decision process; Reserve selection; Stochastic dynamic programming; Uncertainty (search for similar items in EconPapers)
Date: 2013
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0304380013000306
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ecomod:v:254:y:2013:i:c:p:15-21

DOI: 10.1016/j.ecolmodel.2013.01.003

Access Statistics for this article

Ecological Modelling is currently edited by Brian D. Fath

More articles in Ecological Modelling from Elsevier
Bibliographic data for series maintained by Catherine Liu ().