Horseshoe Regularization for Machine Learning in Complex and Deep Models

• Activity Number: 554 - Recent Challenges and Developments in Bayesian Big Data Inference and Computation with Public Database
• Type: Invited
• Date/Time: Thursday, August 6, 2020 : 3:00 PM to 4:50 PM
• Sponsor: ENAR
• Author(s): Anindya Bhadra* and Jyotishka Datta and Yunfan Li and Nicholas Polson
• Companies: Purdue University and University of Arkansas and Purdue University and University of Chicago
• Keywords: complex data; deep learning; large scale machine learning; nonlinear; non-Gaussian; shrinkage

Abstract

Since the advent of the horseshoe priors for regularization, global-local shrinkage methods have proved to be a fertile ground for the development of Bayesian theory and methodology in machine learning. They have achieved remarkable success in computation, and enjoy strong theoretical support. Much of the existing literature has focused on the linear Gaussian case. The purpose of the current talk is to demonstrate that the horseshoe priors are useful more broadly, by reviewing both methodological and computational developments in complex models that are more relevant to machine learning applications. Specifically, we focus on methodological challenges in horseshoe regularization in nonlinear and non-Gaussian models; multivariate models; and deep neural networks. We also outline the recent computational developments in horseshoe shrinkage for complex models along with a list of available software implementations that allows one to venture out beyond the comfort zone of the canonical linear regression problems.