Markov Decision Processes with Recursive Risk Measures

Nicole B\"auerle and Alexander Glauner

Papers from arXiv.org

Abstract: In this paper, we consider risk-sensitive Markov Decision Processes (MDPs) with Borel state and action spaces and unbounded cost under both finite and infinite planning horizons. Our optimality criterion is based on the recursive application of static risk measures. This is motivated by recursive utilities in the economic literature, has been studied before for the entropic risk measure and is extended here to an axiomatic characterization of suitable risk measures. We derive a Bellman equation and prove the existence of Markovian optimal policies. For an infinite planning horizon, the model is shown to be contractive and the optimal policy to be stationary. Moreover, we establish a connection to distributionally robust MDPs, which provides a global interpretation of the recursively defined objective function. Monotone models are studied in particular.

Date: 2020-10
New Economics Papers: this item is included in nep-rmg and nep-upt
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Published in European Journal of Operational Research 2021

Downloads: (external link)
http://arxiv.org/pdf/2010.07220 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2010.07220

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().