Continuous Time Markov Chains

Sivaprasad Madhira and Shailaja Deshmukh ()
Additional contact information
Sivaprasad Madhira: Savitribai Phule Pune University
Shailaja Deshmukh: Savitribai Phule Pune University

Chapter Chapter 6 in Introduction to Stochastic Processes Using R, 2023, pp 321-387 from Springer

Abstract: Abstract This chapter is concerned with the general theory of continuous time discrete state space Markov processes. These processes are also known as Markov pure jump processes or continuous time Markov chains. In Sect. 2, the probabilistic structure of a continuous time Markov chain is discussed. It is proved that the holding times (sojourn times) in the states have exponential distributions. The concept of an embedded Markov chain is also introduced. In Sect. 3, some properties of the transition functions are established. The important concept of the infinitesimal generator of a continuous time Markov chain and its relation to the transition probability matrix of the embedded Markov chain are discussed. Section 5 is concerned with the solutions to Kolmogorov forward and backward equations and also some methods to compute transition functions numerically. In Sect. 6, the long-run behavior of continuous time Markov chains, balance equations and steady-state solutions are discussed. Section 7 contains R codes for computation of transition probability matrix of the embedded chain, for obtaining realizations from a continuous time Markov chain, for computing transition probability function and for computing the stationary and long-run distributions.

Date: 2023
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-981-99-5601-2_6

Ordering information: This item can be ordered from
http://www.springer.com/9789819956012

DOI: 10.1007/978-981-99-5601-2_6

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().