Markov Chains on Continuous State Space

Quan-Lin Li ()
Additional contact information
Quan-Lin Li: Tsinghua University, Department of Industrial Engineering

Chapter 5 in Constructive Computation in Stochastic Models with Applications, 2010, pp 216-287 from Springer

Abstract: Abstract In this chapter, we discuss Markov chains on continuous state space. We first analyze a discrete-time Markov chain on continuous state space, and then discuss a discrete-time Markov chain on a bivariate state space. Applying the censoring technique, we provide expression for the RG-factorizations, which are used to derive the stationary probability of the Markov chain. Further, we consider a continuous-time Markov chain on continuous state space. Specifically, we deal with a continuous-time level-dependent QBD process with continuous phase variable, and provide orthonormal representations for the R-, U- and G-measures, which lead to the matrix-structured computation of the stationary probability. As an application, we introduce continuousphase type (CPH) distribution and continuous-phase Markovian arrival process (CMAP), and then analyze a CMAP/CPH/1 queue. Finally, we study a piecewise deterministic Markov process, which is applied to deal with more general queues such as the GI/G/c queue.

Keywords: Markov chains on continuous state space; QBD process with continuous phase variable; orthonormal representation; continuous-phase type distribution; continuous-phase Markovian arrival process; piecewise deterministic Markov process (search for similar items in EconPapers)
Date: 2010
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-642-11492-2_5

Ordering information: This item can be ordered from
http://www.springer.com/9783642114922

DOI: 10.1007/978-3-642-11492-2_5

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().