Bootstrap Model Averaging in High-Dimensional Regression

• Activity Number: 673
• Type: Topic Contributed
• Date/Time: Thursday, August 13, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Craig Rolling* and Yongli Zhang
• Companies: University of Oregon and University of Oregon
• Keywords: high-dimensional regression ; model averaging ; bootstrap ; LASSO ; bagging

Abstract

While much work has been done in the area of variable selection for high-dimensional regression, less attention has been given to model averaging in high dimensions. Because the high-dimensional setting increases the difficulty of model identification, a well-constructed model average can provide large gains in prediction accuracy over model selection when the number of covariates is large. We introduce a method that uses bootstrap samples to generate the candidate models, then uses a bootstrap-based estimate of prediction error to determine how many models to combine. Unlike traditional bootstrap-based averaging methods (e.g., bagging), our method reduces to model selection in the parametric case. Simulations indicate our method performs well in a variety of high-dimensional settings.