Adaptive and Efficient Parametric Regression in Semi-Supervised Settings

• Activity Number: 610
• Type: Contributed
• Date/Time: Thursday, August 7, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Nonparametric Statistics
• Author(s): Abhishek Chakrabortty*+ and Tianxi Cai
• Companies: Harvard and Harvard
• Keywords: Semi-Supervised Learning ; Parametric Working Models ; Model Misspecification and Model Adaptive Estimation ; Semi-Parametric Augmentation ; Kernel Smoothing ; Dimension Reduction

Abstract

We consider a semi-supervised setting having a large unlabeled data with covariates observed but outcomes unobserved, and a much smaller labeled data with both observed. Such data arises naturally if the outcome, unlike the covariates, is difficult to obtain. We investigate the prospect of using the unlabeled data in training parametric regression models, hoping to improve estimation of the model parameters, compared to supervised approaches using only the labeled data. Considering the linear regression model in particular, we use a semi-parametric augmentation method to construct an estimator that is efficient, and adaptive to model misspecification (hence, more efficient than the supervised one if the working model is misspecified and equally efficient if the model is correct, when no gain is possible anyway without further assumptions). This adaptive property, ensuring no loss under any scenario, is crucial for safely advocating the use of unlabeled data. The estimator can be efficiently used for high dimensional covariates, and can be extended easily to more general settings. We provide theoretical and simulation results for all our claims, and also analyze a real dataset.