Causal Inference in Large-Scale Observational Studies: A Case Study of eBay Mobile App Releases

• Activity Number: 373
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Business and Economic Statistics Section
• Author(s): Vadim von Brzeski* and Matt Taddy and David Draper
• Companies: UC Santa Cruz/eBay Research Labs and The University of Chicago and UC Santa Cruz/eBay Research Labs
• Keywords: causal inference ; observational study ; bayesian modeling

Abstract

Causal inference in observational studies is extremely difficult, due to the fact that the experimenter is not in charge of the treatment assignment mechanism. Many potential confounding factors (PCFs) likely exist; to estimate the causal effect of the treatment, one needs to control for such factors. Finding PCFs may be difficult given a single observational study. However, the task is significantly easier if one observes a sequence of similar treatments over a lengthy time period. We illustrate this method through an analysis of a sequence of observational studies involving app releases from eBay, Inc. We study the behavior of a large set of eBay users over a period of one year to ascertain the set of PCFs that have the most impact on our causal effect estimates. We describe how a sequence of Bayesian models of increasing complexity leads to more realistic estimates of the causal effect. As we investigate long-term patterns of purchasing behavior, we discover PCFs that are highly useful in our models. We then use these models for predicting the counterfactual response, and compare our results to simple regression models.