A Bootstrap Metropolis-Hastings Algorithm for Bayesian Analysis of Big Data

• Activity Number: 511
• Type: Invited
• Date/Time: Wednesday, August 12, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Computing
• Author(s): Faming Liang* and Jinsu Kim and Qifan Song
• Companies: University of Florida and Texas A&M University and Purdue University
• Keywords: Big Data ; Markov Chain Monte Carlo ; Bootstrap ; Metropolis-Hastings Algorithm ; Parallel Computing

Abstract

Markov chain Monte Carlo methods have proven to be a very powerful tool for analyzing data of complex structures. However, their computer-intensive nature, which typically require a large number of iterations and a complete scan of the full dataset for each iteration, precludes their use for big data analysis. We propose the so-called bootstrap Metropolis-Hastings (BMH) algorithm, which provides a general framework for how to tame powerful MCMC methods to be used for big data analysis; that is to replace the full data log-likelihood by a Monte Carlo average of the log-likelihoods that are calculated in parallel from multiple bootstrap samples. The BMH algorithm possesses an embarrassingly parallel structure and avoids repeated scans of the full dataset in iterations, and is thus feasible for big data problems. Compared to the popular divide-and-combine method, BMH can be generally more efficient as it can asymptotically integrate the whole data information into a single simulation run. The BMH algorithm is very flexible. Like the Metropolis-Hastings algorithm, it can serve as a basic building block for developing advanced MCMC algorithms that are feasible for big data problems.