On the Semi-Supervised Joint-Trained Elastic Net

• Activity Number: 284
• Type: Topic Contributed
• Date/Time: Tuesday, August 5, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Computing
• Author(s): Mark Culp*+ and Kenneth J. Ryan
• Companies: West Virginia University and West Virginia University
• Keywords: Machine Learning ; Semi-supervised Learning ; Elastic Net ; Optimization ; Regression

Abstract

Abstract: Most supervised linear regression techniques optimize parameters on the available training feature data and responses (referred to as labeled data) and then use these parameter settings to predict new observations (referred to as unlabeled data). The supervised approach is arguably most successful when the labeled and unlabeled data come from the same distribution. Many practical circumstances do not afford such an assumption, but in many cases the unlabeled data are available at the time of training. Moreover, it is well understood that the variability of predictions on observations outside of the training data range is quite high for linear techniques especially in high dimensional situations. Supervised approaches are at an inherent disadvantage by not accounting for this information.

In this talk, we derive the joint trained elastic net, which specifically addresses this issue using semi-supervised learning. In semi-supervised learning, one is primarily interested in incorporating the full labeled/unlabeled feature data and the labeled responses to improve prediction. We demonstrate geometrically that this approach shrinks the unlabeled fitted predictions in the dir