Variational Inference in Exponential families: Alternatives to MCMC

• Activity Number: 255
• Type: Invited
• Date/Time: Tuesday, August 10, 2004 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): Michael I. Jordan*+
• Companies: University of California, Berkeley
• Keywords: graphical models ; exponential families ; MCMC ; variational methods ; convexity ; bioinformatics

Abstract

Markov chain Monte Carlo (MCMC) has played an important role in statistics in recent years. MCMC is not a panacea, however, and it is important to explore other methods for probabilistic inference, particularly in the setting of models aimed at large-scale data analysis problems. One general approach, like MCMC having its origins in statistical physics, is provided by variational methodology. Variational methods express computations as the solutions to optimization problems, and derive approximations by "relaxations" of this optimization problem. These methods are particularly powerful in the context of exponential family distributions, where tools from convex analysis and convex optimization come into play. They also are particularly natural in the setting of graphical models, where the graphical structure of the model can aid in developing relaxations. I tell a story with three interrelated themes: exponential families, graphical models, and variational inference. I will illustrate these ideas with examples taken from bioinformatics and information retrieval. (Joint work with Martin Wainwright.)