Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 294 - Uncertainty Quantification in Deep Learning
Type: Invited
Date/Time: Wednesday, August 11, 2021 : 3:30 PM to 5:20 PM
Sponsor: Section on Bayesian Statistical Science
Abstract #316908
Title: Consistent Sparse Deep Learning: Theory and Computation
Author(s): Qifan Song* and Faming Liang and Yan Sun
Companies: Purdue University and Purdue University and Purdue University
Keywords: Sparse DNN; Bayesian consistency; generalization bound
Abstract:

Deep learning has been the engine powering many successes of data science. However, the deep neural network (DNN), as the basic model of deep learning, is often excessively over-parameterized, causing many difficulties in training, prediction and interpretation. We propose a frequentist-like method for learning sparse DNNs and justify its consistency under the Bayesian framework: the proposed method could learn a sparse DNN with nice theoretical guarantees. We establish posterior consistency for the sparse DNN with a mixture Gaussian prior, show that the structure of the sparse DNN can be consistently determined using a Laplace approximation-based marginal posterior inclusion probability approach, and use Bayesian evidence to elicit sparse DNNs learned by an optimization method such as stochastic gradient descent in multiple runs with different initializations. The proposed method is computationally more efficient than standard Bayesian methods for large-scale sparse DNNs, and perform very well in large-scale network compression as well as feature selection for high-dimensional nonlinear regression, both advancing interpretable machine learning.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2021 program