Prediction Risk in Linear Regression Models Under Global-Local Mixture Priors

• Activity Number: 641 - Recent Advances in Density Mixture Modeling and EM-Like Algorithms: Frequentist and Bayesian Views
• Type: Topic Contributed
• Date/Time: Thursday, August 1, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Section on Nonparametric Statistics
• Author(s): Anindya Bhadra* and Jyotishka Datta and Yunfan Li and Nicholas Polson and Brandon Willard
• Companies: Purdue University and University of Arkansas and Purdue University and University of Chicago and University of Chicago
• Keywords: horseshoe; shrinkage regression; Stein's unbiased risk estimate; principal components; ridge regression; horseshoe+

Abstract

Predictive performance in shrinkage regression suffers from two major difficulties: (i) the amount of relative shrinkage is monotone in the singular values of the design matrix and (ii) the amount of shrinkage does not depend on the response variables. Both of these factors can translate to a poor prediction performance, the risk of which can be estimated unbiasedly using Stein's approach. We show that using a component-specific local shrinkage term that can be learned from the data under a suitable heavy-tailed prior, in combination with a global term providing shrinkage towards zero, can alleviate both these difficulties and consequently, can result in an improved risk for prediction. Demonstrations of improved prediction performance over competing approaches in a simulation study and in a pharmacogenomics data set confirm our theoretical findings.