Adaptive Nonparametric Regression for Marketplace Response Detection

• Activity Number: 344
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Business and Economic Statistics Section
• Author(s): Junqing Wu*+ and Yuedong Wang and Wendy Meiring
• Companies: Microsoft Corporation and University of California at Santa Barbara and University of California at Santa Barbara
• Keywords: nonparametric regression ; generalized degree of freedom ; analytics ; business intelligence ; data mining ; overcomplete basis

Abstract

In today's technology world, data is increasingly being relied upon for decision-making. Nonparametric techniques have advantages in these scenarios because they don't enforce a lot of structure to the pattern in the data upfront. We have proposed a methodology that is highly adaptive in capturing sudden changes without overfitting. The adaptivity of the methodology comes from the use of multiple "libraries", such as spline and fourier bases, and the parsimony of model representation comes from ad-hoc estimation of model complexity via Monte Carlo simulations. Bootstrap confidence intervals can be obtained simultaneously as a by-product of the computation. It is shown that such a methodology has the potential to make a big impact in A/B testing and marketplace response detection, both of which are common statistical exercises in the technology industry. An R package has been made to implement the methodology.