Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 257 - Online Experimentation at Scale: Challenges and Solutions
Type: Invited
Date/Time: Tuesday, August 4, 2020 : 1:00 PM to 2:50 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #308059
Title: Online Experimentation at Scale: Challenges and Solutions
Author(s): Martin Tingley* and Somit Gupta* and Xiaolin Shi* and Myoungji Lee* and Guillaume Saint-Jacques* and Dennis Sun*
Companies: Netflix and Microsoft and Snap and Lyft and LinkedIN and Cal Poly and Google
Keywords: experimentation; A/B testing; big data; experimental design; data science

Technology companies, large and small, innovate both their internal operations and consumer facing products using online, randomized controlled trials, also known as A/B tests. Scaling A/B testing and other experimentation paradigms to meet the requirements of modern technology companies comes with a wide variety of statistical, computational, educational, and cultural challenges.

This session will bring together well-known practitioners of high frequency and big-data A/B experiments from a number of major technology companies. The discussion will focus broadly on the topic of scale, including the statistical methodology and computational challenges posed by massive data sets and a high throughput of experiments; how to unlock data scientists via platform-level solutions; and associated cultural and educational challenges in an environment of automated, or automation-assisted, decision making.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program