Sparse Regression with Nonsparse Latent Features

• Activity Number: 189
• Type: Contributed
• Date/Time: Monday, August 4, 2014 : 10:30 AM to 12:20 PM
• Sponsor: IMS
• Author(s): Zemin Zheng*+ and Jinchi Lv and Pallavi Basu
• Companies: University of Southern California and University of Southern California and University of Southern California
• Keywords: Sparse regression ; high dimensionality ; latent confounding factors ; principal components ; thresholded regression

Abstract

As a powerful tool for producing interpretable models, sparse modeling has gained popularity for analyzing large-scale data sets. Most of existing methods assume implicitly that all features in a model are observable. Yet some latent confounding factors may exist in the hidden structure of the original model in many applications. Omitting such confounding factors can cause serious issues in both prediction and variable selection. In this paper, we suggest sparse regression with nonsparse latent features that incorporates confounding factors as the principal components of the random design matrix. We estimate the population principal components by the sample ones and establish comprehensive asymptotic properties of estimated confounding factors in high dimensions for a wide class of distributions. With the aid of these properties, we prove that high-dimensional thresholded regression with estimated confounding factors can still enjoy model selection consistency and oracle inequalities under various prediction and variable selection losses for both observable covariates and latent confounding factors. Our new method and results are evidenced by simulation and real data examples.