Spike and Slab Variable Selection: Frequentist and Bayesian Strategies

• Activity Number: 175
• Type: Invited
• Date/Time: Tuesday, August 10, 2004 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Computing
• Author(s): J. Sunil Rao*+ and Hemant Ishwaran
• Companies: Case Western Reserve University and Cleveland Clinic Foundation
• Keywords: generalized ridge regression ; model averaging ; hypervariance ; shrinkage

Abstract

Variable selection in the linear regression model takes many apparent faces from both frequentist and Bayesian standpoints. We introduce a class of variable selection models we refer to as rescaled spike and slab models. We study the importance of the prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation. Specifically, we study the usefulness of continuous priors to model hypervariance parameters, and the effect scaling has on the posterior mean through its relationship to penalization. We demonstrate the importance of selective shrinkage for effective variable selection in terms of risk performance. Using specialized forward and backward selection strategies, we study the effects of selection bias and model uncertainty and compare these to an ordinary least squares (OLS) model estimator formed without the benefit of model averaging. Simulations are used to illustrate the performance of the rescaled spike and slab variable selection technique.