Risk-Sensitive Optimality Criteria in Markov Decision Processes

Karel Sladký ()
Additional contact information
Karel Sladký: Academy of Sciences of the Czech Republic

A chapter in Operations Research Proceedings 2006, 2007, pp 555-561 from Springer

Abstract: Abstract The usual optimization criteria for Markov decision processes (e.g. total discounted reward or mean reward) can be quite insufficient to fully capture the various aspects for a decision maker. It may be preferable to select more sophisticated criteria that also reflect variability-risk features of the problem. To this end we focus attention on risk-sensitive optimality criteria (i.e. the case when expectation of the stream of rewards generated by the Markov processes evaluated by an exponential utility function is considered) and their connections with mean-variance optimality (i.e. the case when a suitable combination of the expected total reward and its variance, usually considered per transition, is selected as a reasonable optimality criterion). The research of risk-sensitive optimality criteria in Markov decision processes was initiated in the seminal paper by Howard and Matheson [6] and followed by many other researchers (see e.g. [1, 2, 3, 5, 4, 8, 9, 14]). In this note we consider a Markov decision chain X = X n, n = 0,1, ... with finite state space $$ \mathcal{I} $$ = 1,2, ..., N and a finite set $$ \mathcal{A}_i $$ = 1,2, ..., K i of possible decisions (actions) in state i ∈ $$ \mathcal{I} $$ . Supposing that in state i ∈ $$ \mathcal{I} $$ action k ∈ $$ \mathcal{A}_i $$ is selected, then state j is reached in the next transition with a given probability p ij k and one-stage transition reward r ij will be accrued to such transition.

Date: 2007
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:oprchp:978-3-540-69995-8_88

Ordering information: This item can be ordered from
http://www.springer.com/9783540699958

DOI: 10.1007/978-3-540-69995-8_88

Access Statistics for this chapter

More chapters in Operations Research Proceedings from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().