Hard Thresholded Regression via Linear Programming

• Activity Number: 39
• Type: Contributed
• Date/Time: Sunday, August 3, 2014 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Qiang Sun*+
• Companies: University of North Carolina at Chapel Hill
• Keywords: Lasso ; correlation bias ; finite sample bias ; sparsity ; hard thresholded regression ; linear programming

Abstract

Inspired by the connection with best subset selection under orthogonal design, we propose the hard thresholded regression for simutaneous variable selection and unbiased estimation in high dimension regime, by identifying potential problems within the non-convex penalized regression. We further extend such framework to ultra-high dimension framework, where the number of covariates may grow at an exponential rate. We propose to incorporate the regularized covariance estimator into the regression procedure to better trade off between noise accumulation and correlation modeling. Hard thresholded regression with regularized covariance matrix includes Sure Independence Screening as a special case. Solid theoretical results have been established. Both simulation and real data results show that our method outperforms other state art of methods.