Markov Decision Processes on Borel Spaces with Total Cost and Random Horizon

Hugo Cruz-Suárez (), Rocio Ilhuicatzi-Roldán () and Raúl Montes- de-Oca ()
Additional contact information
Hugo Cruz-Suárez: Benemérita Universidad Autónoma de Puebla
Rocio Ilhuicatzi-Roldán: Benemérita Universidad Autónoma de Puebla
Raúl Montes- de-Oca: Universidad Autónoma Metropolitana-Iztapalapa

Journal of Optimization Theory and Applications, 2014, vol. 162, issue 1, No 19, 329-346

Abstract: Abstract This paper deals with Markov Decision Processes (MDPs) on Borel spaces with possibly unbounded costs. The criterion to be optimized is the expected total cost with a random horizon of infinite support. In this paper, it is observed that this performance criterion is equivalent to the expected total discounted cost with an infinite horizon and a varying-time discount factor. Then, the optimal value function and the optimal policy are characterized through some suitable versions of the Dynamic Programming Equation. Moreover, it is proved that the optimal value function of the optimal control problem with a random horizon can be bounded from above by the optimal value function of a discounted optimal control problem with a fixed discount factor. In this case, the discount factor is defined in an adequate way by the parameters introduced for the study of the optimal control problem with a random horizon. To illustrate the theory developed, a version of the Linear-Quadratic model with a random horizon and a Logarithm Consumption-Investment model are presented.

Keywords: Markov decision process; Total cost; Random horizon; Varying-time discount factor (search for similar items in EconPapers)
Date: 2014
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10957-012-0262-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:162:y:2014:i:1:d:10.1007_s10957-012-0262-8

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1007/s10957-012-0262-8

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().