Flexible Variational Bayes Based on a Copula of a Mixture of Normals

• Activity Number: 126 - Topics at the Frontier of Statistical Computing and Machine Learning
• Type: Invited
• Date/Time: Monday, August 8, 2022 : 10:30 AM to 12:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Robert Kohn* and David Gunawan and David Nott
• Companies: University of New South Wales and School of Mathematics and Applied Statistics and University of Singapore
• Keywords: Natural-gradient; Multimodal; Stochastic gradient ; Variance reduction

Abstract

Variational Bayes methods approximate the posterior density by a family of tractable distributions and use optimisation to estimate the unknown parameters of the approximation. Variational approximation is useful when exact inference is intractable or very costly. Our article develops a flexible variational approximation based on a copula of a mixture of normals, which is implemented using the natural gradient and a variance reduction method. The efficacy of the approach is illustrated by using simulated and real datasets to approximate multimodal, skewed and heavy-tailed posterior distributions, including an application to Bayesian deep feedforward neural network regression models. Each example shows that the proposed variational approximation is much more accurate than the corresponding Gaussian copula and a mixture of normals variational approximations.