|
Abstract:
|
Both Forward Selection (FS) and the Lasso can perform variable selection in high dimensional regression problems. Although the Lasso is the solution to an optimization problem while FS is purely algorithmic, the two methods utilize surprisingly similar approaches. Both add to the model the variable which has the highest correlation with the residual vector, the only difference being in the level of shrinkage. We propose a new method, Forward-Lasso Adaptive Shrinkage (FLASH), which incorporates both FS and the Lasso as special cases. FLASH is fitted using a variant of the LARS algorithm and works well in situations where neither Lasso nor FS succeeds. We prove that it can be formulated as the solution to a weighted Lasso optimization problem. We also demonstrate, on an extensive set of simulations and a real-world data set, that FLASH generally outperforms many competing approaches.
|
