Time-Sharing Policies for Controlled Markov Chains

Eitan Altman and Adam Shwartz
Additional contact information
Eitan Altman: INRIA, Centre Sophia Antipolis, Valbonne, France
Adam Shwartz: Technion, Haifa, Israel

Operations Research, 1993, vol. 41, issue 6, 1116-1124

Abstract: We propose a class of nonstationary policies called policy time sharing (PTS), which possesses several desirable properties for problems where the criteria are of the average-cost type; an optimal policy exists within this class, the computation of optimal policies is straightforward, and the implementation of this policy is easy. While in the finite state case stationary policies are also known to share these properties, the new policies are much more flexible, in the sense that they can be applied to solve adaptive problems, and they suggest new ways to incorporate the particular structure of the problem at-hand into the derivation of optimal policies. In addition, they provide insight into the pathwise-structure of controlled Markov chains. To use PTS policies one alternates between the use of several stationary deterministic policies, switching when reaching some predetermined state. In some (countable state) cases optimal solutions of the PTS type are available and easy to compute, whereas optimal stationary policies are not available. Examples that illustrate the last point and the usefulness of the new approach are discussed, involving constrained optimization problems with countable state space or compact action space.

Keywords: decision analysis; multiple criteria: Markov decision processes with several constraints; dynamic programming/optimal control; Markov: nonstationary policies; sample path properties (search for similar items in EconPapers)
Date: 1993
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.41.6.1116 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:41:y:1993:i:6:p:1116-1124

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().