ThrEEBoost: Thresholded Boosting for Variable Selection and Prediction via Estimating Equations

• Activity Number: 240
• Type: Contributed
• Date/Time: Monday, August 4, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Julian Wolfson*+ and Christopher Miller
• Companies: University of Minnesota and NAMSA
• Keywords: boosting ; estimating equations ; variable selection ; prediction

Abstract

Most variable selection techniques assume relatively simple settings where observations are independent and completely observed. In contrast, there is a rich literature on approaches to low-dimensional parameter estimation which handle the myriad complexities of ``real'' data, including correlation, missingness, measurement error, selection bias, etc. We present ThrEEBoost, a flexible algorithm for prediction and variable selection based on estimating equations. A generalization of EEBoost (Wolfson, 2011), ThrEEBoost uses a modified boosting technique in which multiple regression coefficients may be updated in a single iteration. The number of coefficients updated at each step is controlled by a thresholding parameter; different thresholding parameter values yield different variable selection paths, and the optimal degree of thresholding can be chosen by cross-validation. We show via simulation that ThrEEBoost performs well in both sparse and non-sparse settings, and offers substantial computational savings over competing L1-regularized methods.