Estimation of High-Dimensional Mean Regression in Absence of Symmetry and Light-Tail Assumptions

• Activity Number: 258
• Type: Contributed
• Date/Time: Monday, August 10, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Yuyan Wang* and Jianqing Fan and Quefeng Li
• Companies: Princeton University and Princeton University and Princeton University
• Keywords: High dimension ; Heavy-tailed & Asymmetric ; Huber loss ; M-estimator ; Optimal rate ; Robust regularization

Abstract

Data subject to heavy-tailed errors are encountered in various scientific fields. Procedures based on quantile regression and Least Absolute Deviation regression have been developed for heavy-tailed data. However, these methods essentially estimate the conditional median function, which can be very different from the conditional mean function especially when the data is asymmetric and heteroscedastic. In this paper, we propose a penalized robust approximate quadratic (RA-quadratic) loss and the RA-Lasso estimator, which estimates the mean regression function assuming only the existence of second moment. We adopt a penalized Huber loss with diverging parameter to reduce the bias of the traditional Huber loss. In the ultra-high dimensional setting where the dimensionality is allowed to grow exponentially with the sample size, we show that RA-Lasso estimator converges at the rate that is optimal even under the light-tail situation. We further show that the composite gradient descent algorithm produces a solution of RA-Lasso that admits the same optimal rate after sufficient iterations. Extensive simulation studies also demonstrate satisfactory finite-sample performance of RA-Lasso.