|
Abstract:
|
Nowadays, the online randomized experiments, aka AB test, are considered as the golden standard to make a ship/no ship decision and measure the impact of a new product feature. However, there are a large amount of scenarios that online randomized experiments may face challenges, especially for online social media. In this talk, we will introduce what type of scenarios that AB tests may fail and what type of remedy we can make at Twitter. Furthermore, we will introduce an extreme case that AB test cannot even be applied. We leveraged a dynamic state space model to forecast the changes; then applied Markov Chain Monte Carlo method to infer the causal impact. The audiences are expected to learn how we run experiments at Twitter, how Twitter uses data science to make product decisions and the data science culture at Twitter as well.
|
