Online Program

Principal Component-Guided Sparse Regression (304031)

 View Presentation

Jerome H Friedman, Stanford University 
*Jingyi Kenneth Tay, Stanford University 
Robert Tibshirani, Stanford University 

Keywords: principal components, sparsity, lasso, supervised learning, feature group selection

We propose a new method for supervised learning, the "principal components lasso" ("pcLasso"). It combines the lasso l1 penalty with a quadratic penalty that shrinks the coefficient vector toward the feature matrix's leading principal components. pcLasso can be especially powerful if the features are pre-assigned to groups. In that case, pcLasso shrinks each group-wise component of the solution toward the leading principal components of that group. In the process, it also carries out selection of feature groups. We provide some theory and illustrate the method on some simulated and real data examples.