Markov Chains

Sivaprasad Madhira and Shailaja Deshmukh ()
Additional contact information
Sivaprasad Madhira: Savitribai Phule Pune University
Shailaja Deshmukh: Savitribai Phule Pune University

Chapter Chapter 2 in Introduction to Stochastic Processes Using R, 2023, pp 31-153 from Springer

Abstract: Abstract Markov chains are widely used discrete time discrete state space stochastic processes. In this chapter we study in sufficient detail the classication of Markov chains which is the first step in analyzing a Markov chain. The basic denitions related to Markov chains are given in the first section. The higher-step transition probabilities and related results are given in Sect. 2. Section 3 deals with generation of realizations (sample paths) of specied length of a Markov chain. This section also contains a brief discussion on the maximum likelihood estimation of the transition probability matrix. The topic of classfiication of the states of a Markov chain, which is essential for the study of long-run behaviour of the Markov chains, is discussed Sects. 4 and 5. An extended discussion and some important results on first passage distributions are given in Sect. 6. Computation of probabilities of absorption into recurring classes, when set of transient states is finite, is also considered in this section. In Section 7, the concept of ‘periodicity’ of states is discussed in detail. In all the sections, the concepts and results are illustrated with computations using R codes given in Sect. 8. These R programs are useful (i) to obtain higher-step transition probabilities (ii) to obtain a finite dimensional distribution (iii) to generate a realization of specified length of a Markov chain (iv) to compute the maximum likelihood estimate of a transition probability matrix (v) to classify the states as persistent or transient (vi) to compute first passage distributions and (vii) to find the period of various states.

Date: 2023
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-981-99-5601-2_2

Ordering information: This item can be ordered from
http://www.springer.com/9789819956012

DOI: 10.1007/978-981-99-5601-2_2

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().