|
Abstract:
|
In this paper we propose estimation via bias corrected least squares after model selection for estimation and variable selection in high dimensional linear regression models with measurement error or missing covariates. We show that separating model selection and estimation leads to an improved rate of convergence of the L2 error compared to the rate sqrt(slog p/n) achieved by simultaneous estimation and variable selection methods such as $\ell_1$ penalized corrected least squares. If the correct model is selected with high probability then the L2 rate of convergence for the proposed method is indeed the oracle rate of sqrt{s/n}. Here s, p, n are the number of nonzero parameters, model dimension and sample size respectively. Under general model selection criteria, the proposed method is computationally simpler and statistically at least as efficient as the $\ell_1$ penalized corrected least squares method, performs model selection without the availability of the bias correction matrix, and is able to provide estimates with only a small sub-block of the bias correction covariance matrix of order s x s in comparison to the p x p correction matrix required for computation of the L1 penalized version. Furthermore we show that the model selection requirements are met by a correlation screening type method and the L1 penalized corrected least squares method. Also, the proposed methodology when applied to the estimation of precision matrices with missing observations, is seen to perform at least as well as existing L1 penalty based methods. All results are supported empirically by a simulation study.
|
