Consistent Variable Selection via Penalized Credible Regions and Confidence Sets

• Activity Number: 615
• Type: Invited
• Date/Time: Thursday, August 8, 2013 : 8:30 AM to 10:20 AM
• Sponsor: JCGS-Journal of Computational and Graphical Statistics
• Author(s): Howard Bondell*+ and Funda Gunes and Brian J. Reich
• Companies: NC State University and SAS Institute and North Carolina State University
• Keywords: adaptive lasso ; confidence region ; credible set ; penalized regression ; stochastic search ; variable selection

Abstract

Selection of relevant predictors for regression is a challenging problem, particularly in high dimensions. Methods such as sure screening, forward selection, or penalized regressions are commonly used. Bayesian variable selection methods place prior distributions on the parameters and on the models. Since exhaustive enumeration is not feasible, posterior model probabilities are often obtained via long MCMC runs. The chosen model can depend heavily on choices for priors and posterior thresholds. Alternatively, we propose a prior only on the parameters in the full model and use sparse solutions within posterior credible regions to perform selection. These credible regions often have closed form representations, and it is shown that these solutions can be computed via existing algorithms. For the case of improper priors with p < n, the credible regions will align with confidence regions, resulting in an approach that resembles an adaptive lasso. In general, the credible set approach is shown to outperform common methods in the high-dimensional setting, particularly under correlation. Consistent model selection of the method is established for both fixed and diverging dimensions.