|
Large-scale A/B experimentation conducted by technology companies (both large and small) has an impact on the daily lives and decisions of billions of people around the globe, as nearly every aspect of the internal operations and consumer experience of companies like Facebook, Google, Amazon, and Netflix have been tested in some manner.
Experimentation in this space operates at vast scale (in terms of both sample sizes and number of tests) and can require rapid decision making. Developing and improving the associated statistical methods for experimental design and analysis is an ongoing topic of applied research at each of the companies represented by the panelists. Example topics include optimal stopping rules for sequential tests, fast identification of negative test experiences, robustness to outliers, Bayesian approaches to A/B testing and overall decision making, and development of statistical methods to better detect small yet meaningful effect sizes amidst noisy data.
This panel session will discuss the current state and limitations of experimentation in this space, and aims to foster more open dialogue and engagement between private sector and academic researchers.
|
