Variable Selection for Big Data via Bagging Adaptive Lasso and Precision Shrinking

• Activity Number: 368
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Cory Lanker*+ and Wen Zhou and Max Morris and Stephen Bruce Vardeman and Huaiqing Wu
• Companies: Iowa State University of Science and Technology and Iowa State University and Iowa State University and Iowa State University and Iowa State University
• Keywords: big data ; variable selection ; lasso ; high dimensionality ; bootstrap ; prediction

Abstract

A communication-efficient algorithm for distributed statistical variable selection is proposed on large-scale data sets. This algorithm is designed to address big data computation limits and estimate variability. Big data sets, those with both large sample size and ultra high dimensionality on the feature space (that is, large n and large p), impose computational difficulties on variable selection algorithms such as Lasso, SCAD, etc. The proposed algorithm is based on a refinement of adaptive Lasso that involves further thresholding on the performance of selected variables on predictions. The algorithm penalizes covariates with high variance and low precision for prediction as estimated by cross validation and bootstrap. The framework is considered for other regularization and thresholding methods as well. Simulation and theoretical justifications are shown and empirical studies based on large data set are used to demonstrate the performance of the proposed algorithm.