Weighted Markov decision processes with perturbation

Ke Liu and Jerzy A. Filar

Mathematical Methods of Operations Research, 2001, vol. 53, issue 3, 465-480

Abstract: In this paper we consider the weighted reward Markov decision process, with perturbation. The “weighted reward” refers to appropriately normalized convex combination of the discounted and the long-run average reward criteria. This criterion allows the controller to trade-off short-term costs versus long-term costs. In every application where both the discounted and the long-run average criteria have been proposed in the past, there is clearly a rationale for considering the weighted criterion. Of course, as with all Markov decision models, the standard weighted criterion model assumes that all the transition probabilities are known precisely. Since, in most applications this would not be the case, we consider the perturbed version of the weighted reward model. We prove that in most cases a nearly optimal control can be found in the class of relatively simple “ultimately deterministic” controls. These are controls which behave just like deterministic stationary controls, after a certain point of time. Copyright Springer-Verlag Berlin Heidelberg 2001

Keywords: Key words: Markov decision processes; weighted reward; optimal policy; δ-optimal; singular perturbation; general perturbation (search for similar items in EconPapers)
Date: 2001
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1007/s001860100125 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:mathme:v:53:y:2001:i:3:p:465-480

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/00186

DOI: 10.1007/s001860100125

Access Statistics for this article

Mathematical Methods of Operations Research is currently edited by Oliver Stein

More articles in Mathematical Methods of Operations Research from Springer, Gesellschaft für Operations Research (GOR), Nederlands Genootschap voor Besliskunde (NGB)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().