|
Abstract:
|
A core problem in statistics is to approximate difficult-to-compute probability distributions. This problem is especially important in Bayesian statistics, which frames all inferences as a calculation about a conditional distribution. Here I review and discuss innovations in variational inference (VI), a method a that uses optimization to approximate probability distributions. VI has been used in myriad applications of Bayesian statistics. It tends to be faster than more traditional methods, such as Markov chain Monte Carlo sampling.
After reviewing the basics, I will discuss some recent research on VI. I describe stochastic variational inference, an approximate inference algorithm for handling massive data sets, and demonstrate its application to probabilistic topic models of millions of articles. Then I discuss black box variational inference, a generic algorithm for approximating the posterior. Black box inference easily applies to many models but requires minimal mathematical work to derive. I demonstrate black box inference on deep exponential families---a method for Bayesian deep learning---and describe how it enables tools for probabilistic programming.
|
