Principal Component-Guided Sparse Regression

• Activity Number: 136 - Recent Advances in Dimension Reduction
• Type: Contributed
• Date/Time: Monday, July 29, 2019 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Kenneth Tay* and Jerome Friedman and Robert Tibshirani
• Companies: Stanford University and Stanford University and Stanford University
• Keywords: principal components; sparsity; lasso; supervised learning; feature group selection

Abstract

We propose a new method for supervised learning, especially suited to wide data where the number of features is much greater than the number of observations. The method combines the lasso l_1 sparsity penalty with a quadratic penalty that shrinks the coefficient vector toward the leading principal components of the feature matrix. We call the proposed method the "principal components lasso" ("pcLasso"). The method can be especially powerful if the features are pre-assigned to groups (such as cell-pathways, assays or protein interaction networks). In that case, pcLasso shrinks each group-wise component of the solution toward the leading principal components of that group. In the process, it also carries out selection of the feature groups.