Global Mean-Field Variational Bayes for Density Regression

• Activity Number: 611 - Nonparametric Priors for Exchangeable Data and Beyond
• Type: Topic Contributed
• Date/Time: Thursday, August 2, 2018 : 8:30 AM to 10:20 AM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Tommaso Rigon* and Daniele Durante
• Companies: Bocconi University and Bocconi University
• Keywords: Bayesian density regression; Logistic regression; Logit stick-breaking; Polya-gamma; Quadratic approximations; Variational Bayes

Abstract

Quadratic approximations of logistic log-likelihoods are fundamental to ease approximate Bayesian inference for binary data. Although the expansions underlying Newton-Raphson scoring methods have attracted much of the interest, there has been also a recent focus on tangent quadratic bounds that uniformly minorize the logistic log-likelihood. A relevant contribution relies on a convex duality argument to derive a tractable family of tangent quadratic expansions. This approximation is still being successfully implemented to facilitate inference in several models, but less attempts have been made to understand the formal reasons underlying its excellent performance. We provide a novel connection between the above bound and a recent Polya-gamma data augmentation. This result places the computational methods associated with the aforementioned bound within a more general and well-established framework (i.e. conjugate global mean-field variational Bayes) having desirable theoretical and computational properties. The practical advantages of such global routines are highlighted in a predictor-dependent mixture of Gaussians for Bayesian nonparametric density regression.