Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions

Rolando Cavazos-Cadena and Emmanuel Fernández-Gaucherand

Mathematical Methods of Operations Research, 1999, vol. 49, issue 2, 299-324

Abstract: We study controlled Markov chains with denumerable state space and bounded costs per stage. A (long-run) risk-sensitive average cost criterion, associated to an exponential utility function with a constant risk sensitivity coefficient, is used as a performance measure. The main assumption on the probabilistic structure of the model is that the transition law satisfies a simultaneous Doeblin condition. Working within this framework, the main results obtained can be summarized as follows: If the constant risk-sensitivity coefficient is small enough, then an associated optimality equation has a bounded solution with a constant value for the optimal risk-sensitive average cost; in addition, under further standard continuity-compactness assumptions, optimal stationary policies are obtained. However, it is also shown that the above conclusions fail to hold, in general, for large enough values of the risk-sensitivity coefficient. Our results therefore disprove previous claims on this topic. Also of importance is the fact that our developments are very much self-contained and employ only basic probabilistic and analysis principles. Copyright Springer-Verlag Berlin Heidelberg 1999

Keywords: Key words: Controlled Markov chains; exponential utility function; constant risk sensitivity; simultaneous Doeblin condition; bounded solutions to the risk-sensitive optimality equation; constant average cost. (search for similar items in EconPapers)
Date: 1999
References: Add references at CitEc
Citations: View citations in EconPapers (15)

Downloads: (external link)
http://hdl.handle.net/10.1007/PL00020919 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:mathme:v:49:y:1999:i:2:p:299-324

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/00186

DOI: 10.1007/PL00020919

Access Statistics for this article

Mathematical Methods of Operations Research is currently edited by Oliver Stein

More articles in Mathematical Methods of Operations Research from Springer, Gesellschaft für Operations Research (GOR), Nederlands Genootschap voor Besliskunde (NGB)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().