Monotonicity properties for two-action partially observable Markov decision processes on partially ordered spaces

Erik Miehling and Demosthenis Teneketzis

European Journal of Operational Research, 2020, vol. 282, issue 3, 936-944

Abstract: This paper investigates monotonicity properties of optimal policies for two-action partially observable Markov decision processes when the underlying (core) state and observation spaces are partially ordered. Motivated by the desirable properties of the monotone likelihood ratio order in imperfect information settings, namely the preservation of belief ordering under conditioning on any new information, we propose a new stochastic order (a generalization of the monotone likelihood ratio order) that is appropriate for when the underlying space is partially ordered. The generalization is non-trivial, requiring one to impose additional conditions on the elements of the beliefs corresponding to incomparable pairs of states. The stricter conditions in the proposed stochastic order reflect a conservation of structure in the problem – the loss of structure from relaxing the total ordering of the state space to a partial order requires stronger conditions with respect to the ordering of beliefs. In addition to the proposed stochastic order, we introduce a class of matrices, termed generalized totally positive of order 2, that are sufficient for preserving the order. Our main result is a set of sufficient conditions that ensures existence of an optimal policy that is monotone on the belief space with respect to the proposed stochastic order.

Keywords: Dynamic programming; Decision analysis; Partially observable Markov decision processes; Partially ordered sets (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0377221719308197
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ejores:v:282:y:2020:i:3:p:936-944

DOI: 10.1016/j.ejor.2019.10.003

Access Statistics for this article

European Journal of Operational Research is currently edited by Roman Slowinski, Jesus Artalejo, Jean-Charles. Billaut, Robert Dyson and Lorenzo Peccati

More articles in European Journal of Operational Research from Elsevier
Bibliographic data for series maintained by Catherine Liu ().