Asymptotics of Variational Inference

• Activity Number: 410
• Type: Topic Contributed
• Date/Time: Tuesday, August 11, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Ted Westling* and Tyler McCormick
• Companies: University of Washington and University of Washington
• Keywords: variational inference ; M-estimation ; asymptotics ; bayesian inference

Abstract

Variational inference is an approximate Bayesian estimation procedure in which recovering the full posterior distribution is traded for finding the Kullback-Leibler projection of the posterior onto a family of tractable approximations. This optimization problem frequently scales well with data size and model complexity, but the resulting approximation does not have the appealing interpretation of a posterior distribution, nor do standard Bayesian asymptotics such as posterior consistency or asymptotic normality apply. There is a growing body of research on properties of variational inference in particular models, but general asymptotic results are yet undeveloped. We will present a general approach for understanding the asymptotics of variational point estimators including consistency and weak convergence utilizing M-estimation theory. Our approach has the advantage that it can be applied to a variety of models, variational families, and "local" variational approximations that do not minimize KL divergence. We demonstrate this broad applicability by constructing valid large-sample confidence sets for variational estimators in a few diverse cases.