Nearly Optimal Controls of Markovian Systems

Q. Zhang, R. H. Liu and G. Yin

Chapter 2 in Stochastic Modeling and Optimization, 2003, pp 43-86 from Springer

Abstract: Abstract This chapter is concerned with asymptotically optimal controls of Markovian systems with weak and strong interactions in discrete time. The primary motivation of our study stems from many applications in operations research such as resource allocation, queueing networks, machine replacement, and command control. Due to the complexity of the real-world problems, one often has to deal with large dimensional systems under uncertainty. Frequently, the underlying Markov chain displays jump behavior having different rates of jumps (some of them vary fast, whereas others change slowly). It is natural to divide the states of such Markov chain into several groups so that transitions among the states within each group occur much more frequently than the transitions among the states belonging to different groups. The basic idea of our approach is to aggregate the states in each group as a single state and to derive a limit problem of smaller dimension. Using an optimal control of the limit problem, we construct controls of the original problem, leading to asymptotic optimality. Our hierarchical approach provides an efficient tool for treating large and complex systems and for finding approximately optimal solutions.

Keywords: Markov Chain; Riccati Equation; Limit Problem; Linear Quadratic; Asymptotic Optimality (search for similar items in EconPapers)
Date: 2003
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-0-387-21757-4_2

Ordering information: This item can be ordered from
http://www.springer.com/9780387217574

DOI: 10.1007/978-0-387-21757-4_2

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().