Computation of the steady-state probability of Markov chain evolving on a mixed state space

Zakrad Az-eddine () and Nasroallah Abdelaziz ()
Additional contact information
Zakrad Az-eddine: Department of Mathematics, Semlalia Faculty of Sciences, Cadi Ayyad University, Marrakech, Morocco
Nasroallah Abdelaziz: Department of Mathematics, Semlalia Faculty of Sciences, Cadi Ayyad University, Marrakech, Morocco

Monte Carlo Methods and Applications, 2023, vol. 29, issue 3, 259-274

Abstract: The partitioning algorithm is an iterative procedure that computes explicitly the steady-state probability of a finite Markov chain 𝑋. In this paper, we propose to adapt this algorithm to the case where the state space E := C ∪ D E:=C\cup D is composed of a continuous part 𝐶 and a finite part 𝐷 such that C ∩ D = ∅ C\cap D=\emptyset . In this case, the steady-state probability 𝜋 of 𝑋 is a convex combination of two steady-state probabilities π C \pi_{C} and π D \pi_{D} of two Markov chains on 𝐶 and 𝐷 respectively. The obtained algorithm allows to compute explicitly π D \pi_{D} . If π C \pi_{C} cannot be computed explicitly, our algorithm approximates it by numerical resolution of successive integral equations. Some numerical examples are studied to show the usefulness and proper functioning of our proposal.

Keywords: Markov chain; steady-state probability; coupling from the past; mixed CFTP; perfect simulation (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1515/mcma-2023-2003 (text/html)
For access to full text, subscription to the journal or payment for the individual article is required.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:mcmeap:v:29:y:2023:i:3:p:259-274:n:6

Ordering information: This journal article can be ordered from
https://www.degruyter.com/journal/key/mcma/html

DOI: 10.1515/mcma-2023-2003

Access Statistics for this article

Monte Carlo Methods and Applications is currently edited by Karl K. Sabelfeld

More articles in Monte Carlo Methods and Applications from De Gruyter
Bibliographic data for series maintained by Peter Golla ().