Bayesian Penalized Regression

• Activity Number: 244
• Type: Contributed
• Date/Time: Monday, August 10, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Ding Xiang* and Galin Jones
• Companies: University of Minnesota and University of Minnesota
• Keywords: MCMC ; Gibbs sampler ; Metropolis-Hastings ; Bayesian bridge regression

Abstract

The well known lasso and ridge regression are special cases of a more general penalized regression family, called the bridge regression and firstly proposed by Frank and Friedman (1993), with the summation of alpha-powered absolute-value norm of coefficients as the penalty function, where alpha can take values between 0 and 2. when alpha equals 2 it corresponds to ridge regression while if alpha is equal to 1 lasso is recovered. When alpha goes to 0, it achieves the subset selection. To choose alpha, classical frequentist way is to set up a candidate set and then achieved via cross validation. Our goal is to propose a data-based, fully Bayesian way to solve this problem. Prior distributions are assigned to every parameter. Metropolis-Hastings algorithm and Gibbs sampler are implemented to find the posterior mean of betas and select the alpha. Comparison in terms of predictions among this new Bayesian approach, the lasso, and ridge regression is made through both simulation and real data study. It is shown that this new method can choose alphas which correspond to frenquentist's experiences in some situations and has better prediction performance in most settings.