Scalable Inference for Hierarchical Topic Models

• Activity Number: 621
• Type: Invited
• Date/Time: Thursday, August 8, 2013 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Computing
• Author(s): John W. Paisley*+
• Companies: University of California, Berkeley
• Keywords: Stochastic variational inference ; Hierarchical topic modeling

Abstract

An important challenge in using Bayesian models for machine learning applications is scaling inference up to large data sets. For example, in models of text, where the number of documents can be in the millions, traditional sampling methods are prohibitively slow. An alternative to sampling is mean-field variational inference, which approximates the posterior with a factorized distribution by attempting to minimize their KL divergence. Recent applications of stochastic optimization techniques to this inference approach has allowed for the scaling up of Bayesian inference to massive data sets. This talk will review this stochastic variational inference approach, and discuss its application to the problem of hierarchical topic modeling using two million documents from The New York Times.

(Joint work with Chong Wang, David Blei and Michael I. Jordan)