Abstract:
|
High-dimensional linear prediction, where the number of features is larger than the number of observations, plays an important role in fields such as genetics and natural language processing. Existing sparse methods for solving such problems have been stunningly successful, but some open questions remain. Often, these methods can fail to select the correct predictors, predict poorly relative to non-sparse alternatives, ignore the unknown grouping structures for the features, or have poor computational tractability. We propose a method called sparsePC to handle high-dimensional prediction tasks including regression and classification, especially in the context of linearly related predictors. SparsePC first estimates sparse principal components and then estimates a linear model on the recovered subspace. Because the estimated subspace is sparse, the resulting predictions will depend on only a small set of features. Our method works well in a variety of simulated and real data examples, performs nearly optimally if the modeling assumptions are satisfied, and possessing near-optimal theoretical guarantees.
|