Markov Chain Design Problems

Richard V. Evans
Additional contact information
Richard V. Evans: University of Illinois, Urbana-Champaign, Illinois

Operations Research, 1981, vol. 29, issue 5, 959-970

Abstract: System design problems using Markov chains and denoted Markov chain design problems are introduced. It is assumed that the chains of the problem have transition matrices and one period conditional expected rewards which are differentiable functions of a vector parameter. A gradient algorithm for maximizing the sum of the discounted expected reward or the limit of the one period expected reward is presented. The algorithm uses approximate objective function values and approximate gradients at each stage. Numerical work on a simple one dimensional queueing model solved the design problem correctly and quickly even though minimal approximation accuracy was used. The convergence of the approximation is proven for an appropriately converging sequence of design parameter values.

Keywords: 696 system design problem; 642 computing an optimal Markov chain (search for similar items in EconPapers)
Date: 1981
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.29.5.959 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:29:y:1981:i:5:p:959-970

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().