Speeding up MCMC by Efficient Data Subsampling

• Activity Number: 486
• Type: Topic Contributed
• Date/Time: Wednesday, August 6, 2014 : 10:30 AM to 12:20 PM
• Sponsor: International Society for Bayesian Analysis (ISBA)
• Author(s): Mattias Villani*+ and Matias Quiroz and Robert J. Kohn
• Companies: Linköping University and Sveriges Riksbank/Stockholm University and University of New South Wales
• Keywords: Bayesian ; Pseudo-marginal MCMC ; Subsampling ; Big Data ; Multivariate probit

Abstract

The computing time for Markov Chain Monte Carlo (MCMC) algorithms can be prohibitively large for datasets with many observations, especially when the data density for each observation is costly to evaluate. We propose a framework based on a Pseudo-marginal MCMC where the likelihood function is unbiasedly estimated from a random subset of the data, resulting in substantially fewer density evaluations. The subsets are selected using efficient sampling schemes, such as Probability Proportional-to-Size (PPS) sampling where the inclusion probability of an observation is proportional to an approximation of its contribution to the likelihood function. We illustrate the method on a large dataset of Swedish firms containing half a million observations.