Optimal Control of a Continuous-Time Markov Chain with Periodic Transition Probabilities

Anders Martin-Löf
Additional contact information
Anders Martin-Löf: The Royal Institute of Technology, Stockholm, Sweden

Operations Research, 1967, vol. 15, issue 5, 872-881

Abstract: Controlled stationary Markov chains with a finite number of states and possible actions and continuous time have been studied by many authors. It is well known that a control maximizing the expected reward in a finite time interval can be obtained from the equation of dynamic programming, and that there exists a stationary control giving maximal (discounted or average) reward in an infinite time interval. In this paper controlled Markov chains whose transition probabilities are periodic in time are considered. It is shown that there exists a periodic control giving maximal expected future reward, and that it can be approximatively calculated by making a discretization in time. An example of a system that can be described by such a process is a service system or an inventory whose input is, e.g., a Poisson process with an intensity that varies seasonally.

Date: 1967
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.15.5.872 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:15:y:1967:i:5:p:872-881

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().