Abstract:
|
Inspired by the architectures of deep Bayesian neural networks (BNN), the deep Gaussian process (DGP) framework has arisen as a way to leverage composition to provide a flexible model that admits tractable Bayesian inference. Recently, the connection between infinitely wide Bayesian neural networks with finite bottleneck layers and DGPs has been established. This correspondence hints at a novel way to approach BNNs, by using the inducing point framework commonly used to train Gaussian process models, as opposed to conventional parameter-centric approaches to BNNs such as Bayes-by-Backprop. In this work, we adopt this perspective to design a fully Bayesian inference method for a shallow convolutional neural network designed for natural language processing (NLP) tasks, and show its practicality on common datasets. Our method employs both inducing points and parametric variational methods, owing to the unique structure of the embedding layer found in NLP networks.
|