Ambiguous partially observable Markov decision processes: Structural results and applications

Soroush Saghafian

Journal of Economic Theory, 2018, vol. 178, issue C, 1-35

Abstract: Markov Decision Processes (MDPs) have been widely used as invaluable tools in dynamic decision-making, which is a central concern for economic agents operating at both the micro and macro levels. Often the decision maker's information about the state is incomplete; hence, the generalization to Partially Observable MDPs (POMDPs). Unfortunately, POMDPs may require a large state and/or action space, creating the well-known “curse of dimensionality.” However, recent computational contributions and blindingly fast computers have helped to dispel this curse. This paper introduces and addresses a second curse termed “curse of ambiguity,” which refers to the fact that the exact transition probabilities are often hard to quantify, and are rather ambiguous. For instance, for a monetary authority concerned with dynamically setting the inflation rate so as to control the unemployment, the dynamics of unemployment rate under any given inflation rate is often ambiguous. Similarly, in worker-job matching, the dynamics of worker-job match/proficiency level is typically ambiguous. This paper addresses the “curse of ambiguity” by developing a generalization of POMDPs termed Ambiguous POMDPs (APOMDPs), which not only allows the decision maker to take into account imperfect state information, but also tackles the inevitable ambiguity with respect to the correct probabilistic model of transitions.

Keywords: POMDP; Unknown probabilities; Model ambiguity; Structural results; Control-limit policies (search for similar items in EconPapers)
JEL-codes: C61 D81 D83 D84 (search for similar items in EconPapers)
Date: 2018
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (10)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0022053118304770
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:jetheo:v:178:y:2018:i:c:p:1-35

DOI: 10.1016/j.jet.2018.08.006

Access Statistics for this article

Journal of Economic Theory is currently edited by A. Lizzeri and K. Shell

More articles in Journal of Economic Theory from Elsevier
Bibliographic data for series maintained by Catherine Liu ().