Robust Variable Selection with a Multiple Step Bootstrap Procedure

• Activity Number: 656
• Type: Contributed
• Date/Time: Thursday, August 13, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Marie-Helene Roy* and Denis Larocque and Debbie Dupuis
• Companies: HEC Montreal and HEC Montreal and HEC Montreal
• Keywords: Variable selection ; Bootstrap ; Model averaging ; VIF ; Robust regression

Abstract

We propose a robust variable selection method based on the Robust VIF (RobVIF) developed by Dupuis and Victoria-Feser (2012). Their proposed method, inspired by the classical VIF streamwise regression, is a very fast, robust, and effective variable selection method. It is, however, sensitive to the initial variable order. Therefore, to correct this sensitivity and to further robustify and enhance feature selection performance while keeping its good properties, we developed a multiple step bootstrap method. In order to reach a high and consistent performance with a large variety of data settings, including a large number of variables, the data is put through a series of bootstrapped RobVIF's. This enables a gradual elimination of unwanted variables. At each step, with a new look at the data, the selection is further refined. The procedure facilitates the identification of superfluous variables while preventing the elimination of the true model predictors. Our method was tested via a simulation, with both clean and contaminated scenarios, and on real data sets. The results show very good, robust and consistent variable selection performance with a wide array of data.