SDA-Bayes and MAD-Bayes for Large-Scale Bayesian Analysis

• Activity Number: 154
• Type: Topic Contributed
• Date/Time: Monday, August 4, 2014 : 10:30 AM to 12:20 PM
• Sponsor: International Society for Bayesian Analysis (ISBA)
• Author(s): Michael Jordan*+
• Companies: University of California, Berkeley
• Keywords: variational methods ; streaming data ; distributed computing ; Bayesian computation

Abstract

We present two lines of research aimed at Bayesian analysis of very large data sets. The first, SDA-Bayes, is a framework for (S)treaming, (D)istributed, (A)synchronous computation of a Bayesian posterior. The framework makes streaming updates to an approximate posterior according to a user-specified approximation batch primitive. We demonstrate the usefulness of our framework, with variational Bayes as the primitive, by fitting the latent Dirichlet allocation model to two large-scale document collections. The second, MAD-Bayes, is based on taking a small-variance asymptotic expansion of the posterior, yielding fast deterministic algorithms that are in the spirit of the classical K-means algorithm for clustering, but taking advantage of the flexibility of Bayesian nonparametric priors. [Joint work with Nick Boyd, Tamara Broderick, Brian Kulis, Andre Wibisono and Ashia Wilson.]