Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 204 - Experimental Design
Type: Contributed
Date/Time: Tuesday, August 4, 2020 : 10:00 AM to 2:00 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #313110
Title: How Twitter Makes Causal Inference If AB Test Fails
Author(s): Wutao Wei*
Companies: Twitter
Keywords: Online Randomized Experiment; Twitter; State Space Model; Causal Inference; Data Science; Forecasting

Nowadays, the online randomized experiments, aka AB test, are considered as the golden standard to make a ship/no ship decision and measure the impact of a new product feature. However, there are a large amount of scenarios that online randomized experiments may face challenges, especially for online social media. In this talk, we will introduce what type of scenarios that AB tests may fail and what type of remedy we can make at Twitter. Furthermore, we will introduce an extreme case that AB test cannot even be applied. We leveraged a dynamic state space model to forecast the changes; then applied Markov Chain Monte Carlo method to infer the causal impact. The audiences are expected to learn how we run experiments at Twitter, how Twitter uses data science to make product decisions and the data science culture at Twitter as well.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program