Risk-Averse Markov Decision Processes Through a Distributional Lens

Ziteng Cheng () and Sebastian Jaimungal ()
Additional contact information
Ziteng Cheng: Department of Statistical Sciences, University of Toronto, Toronto, Ontario M5G 1Z5, Canada
Sebastian Jaimungal: Department of Statistical Sciences, University of Toronto, Toronto, Ontario M5G 1Z5, Canada

Mathematics of Operations Research, 2025, vol. 50, issue 3, 1707-1733

Abstract: By adopting a distributional viewpoint on law-invariant convex risk measures, we construct dynamic risk measures (DRMs) at the distributional level. We then apply these DRMs to investigate Markov decision processes, incorporating latent costs, random actions, and weakly continuous transition kernels. Furthermore, the proposed DRMs allow risk aversion to change dynamically. Under mild assumptions, we derive a dynamic programming principle and show the existence of an optimal policy in both finite and infinite time horizons. Moreover, we provide a sufficient condition for the optimality of deterministic actions. For illustration, we conclude the paper with examples from optimal liquidation with limit order books and autonomous driving.

Keywords: 90C39; 91G70; dynamic programming; Markov decision processes; risk measures; risk averse (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/moor.2023.0211 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormoor:v:50:y:2025:i:3:p:1707-1733

Access Statistics for this article

More articles in Mathematics of Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().