UPS Delivers Optimal Phase Diagram in High-Dimensional Variable Selection

• Activity Number: 651
• Type: Contributed
• Date/Time: Thursday, August 8, 2013 : 8:30 AM to 10:20 AM
• Sponsor: IMS
• Author(s): Pengsheng Ji*+ and Jiashun Jin
• Companies: University of Georgia and Carnegie Mellon University
• Keywords: variable selection ; Screen and Clean ; Lasso

Abstract

Consider the linear regression model Y=X_{n,p} \beta + z where both p and n are large but p>>n. We find for the setting where both the signal beta and the Gram matrix X'X are sparse, neither the lasso nor the subset selection is asymptotically optimal in terms of the Hamming errors, or the phase diagram. We propose the Univariate Penalization Screening (UPS) for variable selection. This is a screen and clean method where we screen with univariate thresholding, and clean with penalized MLE. It has two important properties: sure screening and separable after screening. These properties enable us to reduce the original regression problem to many small-size regression problems that can be fitted separately. The UPS is effective both in theory and in computation.