Derivative Estimates from Simulation of Continuous-Time Markov Chains

Paul Glasserman
Additional contact information
Paul Glasserman: Columbia University, New York, New York

Operations Research, 1992, vol. 40, issue 2, 292-308

Abstract: Countable-state, continuous-time Markov chains are often analyzed through simulation when simple analytical expressions are unavailable. Simulation is typically used to estimate costs or performance measures associated with the chain and also characteristics like state probabilities and mean passage times. Here we consider the problem of estimating derivatives of these types of quantities with respect to a parameter of the process. In particular, we consider the case where some or all transition rates depend on a parameter. We derive derivative estimates of the infinitesimal perturbation analysis type for Markov chains satisfying a simple condition, and argue that the condition has significant scope. The unbiasedness of these estimates may be surprising—a “naive” estimator would fail in our setting. What makes our estimates work is a special construction of specially structured parameteric families of Markov chains. In addition to proving unbiasedness, we consider a variance reduction technique and make comparisions with derivative estimates based on likelihood ratios.

Keywords: simulation; statistical analysis of derivation estimates; simulation; Markov processes: sensitivity analysis for (search for similar items in EconPapers)
Date: 1992
References: Add references at CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://dx.doi.org/10.1287/opre.40.2.292 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:40:y:1992:i:2:p:292-308

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().