SPEEDING UP MCMC BY EFFICIENT DATA SUBSAMPLING

Matias Quiroz (), Mattias Villani and Robert Kohn ()
Additional contact information
Matias Quiroz: Research Department, Central Bank of Sweden, Postal: Sveriges Riksbank, SE-103 37 Stockholm, Sweden

No 297, Working Paper Series from Sveriges Riksbank (Central Bank of Sweden)

Abstract: The computing time for Markov Chain Monte Carlo (MCMC) algorithms can be prohibitively large for datasets with many observations, especially when the data density for each observation is costly to evaluate. We propose a framework where the likelihood function is estimated from a random subset of the data, resulting in substantially fewer density evaluations. The data subsets are selected using an efficient Probability Proportional-to-Size (PPS) sampling scheme, where the inclusion probability of an observation is proportional to an approximation of its contribution to the log-likelihood function. Three broad classes of approximations are presented. The proposed algorithm is shown to sample from a distribu- tion that is within O(m^-1/2) of the true posterior, where m is the subsample size. Moreover, the constant in the O(m^-1/2) error bound of the likelihood is shown to be small and the approximation error is demonstrated to be negligible even for a small m in our applications. We propose a simple way to adaptively choose the sample size m during the MCMC to optimize sampling efficiency for a fixed computational budget. The method is applied to a bivariate probit model on a data set with half a million observations, and on a Weibull regression model with random effects for discrete-time survival data.

Keywords: Bayesian inference; Markov Chain Monte Carlo; Pseudo-marginal MCMC; Big Data; Probability Proportional-to-Size sampling; Numerical integration. (search for similar items in EconPapers)
JEL-codes: C11 C13 C15 C55 C83 (search for similar items in EconPapers)
Pages: 46 pages
Date: 2015-03-01
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.riksbank.se/Documents/Rapporter/Working ... rap_wp297_150330.pdf (application/pdf)

Related works:
Journal Article: Speeding Up MCMC by Efficient Data Subsampling (2019)
Working Paper: Speeding up MCMC by Efficient Data Subsampling (2016)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hhs:rbnkwp:0297

Access Statistics for this paper

More papers in Working Paper Series from Sveriges Riksbank (Central Bank of Sweden) Sveriges Riksbank, SE-103 37 Stockholm, Sweden. Contact information at EDIRC.
Bibliographic data for series maintained by Lena Löfgren ().