Data Enrichment for Linear Regression Models

• Activity Number: 399
• Type: Invited
• Date/Time: Tuesday, August 6, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Aiyou Chen and Art B. Owen*+ and Minghui Shi
• Companies: Google Inc. and Stanford University and Google Inc.
• Keywords: Stein shrinkage ; Small area estimation ; Transfer learning

Abstract

Even in the age of big data, small data sets remain relevant. One problem faced by Internet companies is combining small data sets with high quality but high cost observations (e.g. a panel recruited by a probability sample), with larger data sources such as log files. More generally, one source may have lower bias while the other has lower variance. Regressions in the large data set may be similar though not identical to those in the smaller one. We address this problem via Stein shrinkage between the two data sets where the goal is to predict in the small data set. The method generalizes small area estimation from survey sampling, and is also an example of transfer learning. For linear regression we give conditions under which simply using the small data set is inadmissible no matter how large the bias between the two populations. The improvement sets in at dimension 5, not 3 which arises in Stein shrinkage of means. We also look at L1 alternatives to Stein shrinkage. Art Owen contributed to this work as a consultant for Google, and not as part of his Stanford duties.