Estimating Monte Carlo variance from multiple Markov chains

Kushagra Gupta and Dootika Vats

Scandinavian Journal of Statistics, 2026, vol. 53, issue 1, 335-363

Abstract: Modern computational advances have enabled easy parallel implementations of Markov chain Monte Carlo (MCMC). However, almost all work in estimating the variance of Monte Carlo averages, including the efficient batch means (BM) estimator, focuses on a single‐chain MCMC run. We demonstrate that simply averaging covariance matrix estimators from multiple chains can yield critical underestimates in small Monte Carlo sample sizes, especially for slow‐mixing Markov chains. We extend the work of Argon & Andradóttir (2006) and propose a multivariate replicated batch means (RBM) estimator that utilizes information from parallel chains, thereby correcting for the underestimation. Under weak conditions on the mixing rate of the process, RBM is strongly consistent and exhibits similar large‐sample bias and variance to the BM estimator. We also exhibit superior theoretical properties of RBM by showing that the (negative) bias in the RBM estimator is less than that of the average BM estimator in the presence of positive correlation in MCMC. Consequently, in small runs, the RBM estimator can be dramatically superior, and this is demonstrated through a variety of examples.

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/sjos.70036

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:scjsta:v:53:y:2026:i:1:p:335-363

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0303-6898

Access Statistics for this article

Scandinavian Journal of Statistics is currently edited by ÿrnulf Borgan and Bo Lindqvist

More articles in Scandinavian Journal of Statistics from Danish Society for Theoretical Statistics, Finnish Statistical Society, Norwegian Statistical Association, Swedish Statistical Association
Bibliographic data for series maintained by Wiley Content Delivery ().