The Randomization Technique as a Modeling Tool and Solution Procedure for Transient Markov Processes

Donald Gross and Douglas R. Miller
Additional contact information
Donald Gross: The George Washington University, Washington, D.C.
Douglas R. Miller: The George Washington University, Washington, D.C.

Operations Research, 1984, vol. 32, issue 2, 343-361

Abstract: We present a randomization procedure for computing transient solutions to discrete state space, continuous time Markov processes. This procedure computes transient state probabilities. It is based on a construction relating a continuous time Markov process to a discrete time Markov chain. Modifications and extensions of the randomization method allow for computation of distributions of first passage times and sojourn times in Markov processes, and also the computation of expected cumulative occupancy times and expected number of events occurring during a time interval. Several implementations of the randomization procedure are discussed. In particular we present an implementation for a general class of Markov processes that can be described in terms of state space ( S ), event set ( E ), rate vectors ( R ), and target vectors ( T )—abbreviated as SERT . This general approach can handle systems whose state spaces are quite large, if they have sparse generators.

Keywords: 704; transient; solutions; for; Markov; processes (search for similar items in EconPapers)
Date: 1984
References: Add references at CitEc
Citations: View citations in EconPapers (12)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.32.2.343 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:32:y:1984:i:2:p:343-361

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().