Bayesian Dimension Reduction Using Neural Networks

• Activity Number: 585 - Bayesian Neural Networks
• Type: Topic Contributed
• Date/Time: Thursday, August 6, 2020 : 3:00 PM to 4:50 PM
• Sponsor: International Society for Bayesian Analysis (ISBA)
• Author(s): Deborshee Sen* and David Dunson and Theodore Papamarkou
• Companies: Duke University and Duke University and Oak Ridge National Laboratory
• Keywords: Bayesian; Dimension reduction; Generative model; Neural network

Abstract

In conducting non-linear dimensionality reduction and feature learning, it is common to suppose that the data lie near a lower-dimensional manifold. One class of model-based approaches for such problems includes latent variables in an unknown non-linear regression function; this includes Gaussian process latent variable models (GP-LVMs) and variational auto-encoders (VAEs) as special cases. VAEs use neural networks and additionally employs approximations to make the computation tractable; however, current implementations lack adequate uncertainty quantification in estimating the unknown density and the lower-dimensional subspace, and can be unstable and lack reproducibility in practice. We attempt to solve this problem by designing Markov chain Monte Carlo (MCMC) sampling algorithms for fully Bayesian inferences in neural network models with latent variables. We address issues of identifiability by imposing constraints on the neural network parameters as well as by using anchor points.