Speeding up MCMC by Efficient Data Subsampling

Robert Kohn (), Matias Quiroz, Minh-Ngoc Tran and Mattias Villani

Working Papers from University of Sydney Business School, Discipline of Business Analytics

Abstract: We propose Subsampling MCMC, a Markov Chain Monte Carlo (MCMC) framework where the likelihood function for n observations is estimated from a random subset of m observations. We introduce a general and highly efficient unbiased estimator of the log-likelihood based on control variates obtained from clustering the data. The cost of computing the log-likelihood estimator is much smaller than that of the full log-likelihood used by standard MCMC. The likelihood estimate is bias-corrected and used in two correlated pseudo-marginal algorithms to sample from a perturbed posterior, for which we derive the asymptotic error with respect to n and m, respectively. A practical estimator of the error is proposed and we show that the error is negligible even for a very small m in our applications. We demonstrate that Subsampling MCMC is substantially more efficient than standard MCMC in terms of sampling efficiency for a given computational budget, and that it outperforms other subsampling methods for MCMC proposed in the literature.

Keywords: Survey sampling; Big Data; Block pseudo-marginal; Estimated likelihood; Correlated pseudo-marginal; Bayesian inference (search for similar items in EconPapers)
Date: 2016
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/2123/16205

Related works:
Journal Article: Speeding Up MCMC by Efficient Data Subsampling (2019)
Working Paper: SPEEDING UP MCMC BY EFFICIENT DATA SUBSAMPLING (2015)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:syb:wpbsba:2123/16205

Access Statistics for this paper

More papers in Working Papers from University of Sydney Business School, Discipline of Business Analytics Contact information at EDIRC.
Bibliographic data for series maintained by Artem Prokhorov ().