Robust estimation of a Markov chain transition matrix from multiple sample paths

Lasse Leskelä and Maximilien Dreveton

Statistica Neerlandica, 2026, vol. 80, issue 1

Abstract: Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and stationary distribution from observed sample paths is a core statistical challenge, particularly when multiple independent trajectories are available. While classical theory typically assumes identical chains with known stationary distributions, real‐world data often arise from heterogeneous chains whose transition kernels and stationary measures might differ from a common target. We analyze empirical estimators for such parallel Markov processes and establish sharp concentration inequalities that generalize Bernstein‐type bounds from standard time averages to ensemble‐time averages. Our results provide nonasymptotic error bounds and consistency guarantees in high‐dimensional regimes, accommodating sparse or weakly mixing chains, model mismatch, nonstationary initializations, and partially corrupted data. These findings offer rigorous foundations for statistical inference in heterogeneous Markov chain settings common in modern computational applications.

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/stan.70023

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:stanee:v:80:y:2026:i:1:n:e70023

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0039-0402

Access Statistics for this article

Statistica Neerlandica is currently edited by Miroslav Ristic, Marijtje van Duijn and Nan van Geloven

More articles in Statistica Neerlandica from Netherlands Society for Statistics and Operations Research
Bibliographic data for series maintained by Wiley Content Delivery ().