Operationalizing Massively Multicore Computation for Bayesian MCMC and Stochastic Search: Clusters, Threads, Cores and GPUs

• Activity Number: 5
• Type: Invited
• Date/Time: Sunday, August 1, 2010 : 2:00 PM to 3:50 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Quanli Wang*+ and Marc Suchard and Andrew J. Cron and Cliburn Chan and Mike West
• Companies: Duke University and University of California, Los Angeles and Duke University and Duke University and Duke University

Abstract

We present case studies on the utility of Graphics Processing Units (GPUs) to perform massively parallel computations of Bayesian mixture models on large datasets. GPUs are relatively cheap computational devices that can be housed in desktop even laptop computers and have the potential to speed up algorithms by 100 fold. We demonstrate that its Single-Instruction-Multiple-Data architecture gives it added advantage over conventional distributed computer clusters for both EM algorithm and Gibbs-sampler based MCMC algorithm for Bayesian mixture models. The very major speedups achievable, based on our developments, for mixture models with larger numbers of components and with large data sets (n=10^5-10^7) support the view that GPU-based computing can and should become very widely utilized for desktop-based, parallel computation for simulation and optimization statistical computations.