Deconstruction of Effects by Exposure Dose

• Activity Number: 133
• Type: Contributed
• Date/Time: Monday, August 4, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Quality and Productivity Section
• Author(s): William Heavlin*+
• Companies: Google
• Keywords: big data ; clickstream data ; Frisch-Waugh-Lovell theorem ; numerator-denominator plots ; ratio statistics ; regression diagnostics

Abstract

 This paper presents an approach to inferential modeling enabled by large data volumes, one combining the elements of experimental design, data stratification, modeling, and visualization.  The application considers searches on google.com, drawing data from experiments changing certain details of how google.com replies to user queries. The goal is to learn how users adapt to such changes.  Stratifying users by their experimental exposure (or "dose"), the response measures how users change over the longer term ("user learning"). Covariates consist of user behaviors measured before the experiments begin (the"pre-period").  The approach centers on a family of linear regression models parametrized by dose. We deconstruct the ratio estimates of multiple regression coefficients by numerator-denominator (ND) plots. Such plots, which generalize cusum charts, have close connections to standard regression diagnostic plots. The conclusions focus ultimately on experimentally induced coefficient shifts, shifts which correspond to the sensitization or desensitization with respect to previous user experiences. These shifts are visualized by ND-plots of a certain Mantel-Haenszel-type ratio.