|
Abstract:
|
A/B testing (randomized controlled trials with a completely randomized design involving a single treatment group (A) and a single control group (B)) is widely employed by tech companies, to improve the design of the web experience offered to customers and for other purposes. As simple as this approach is from the viewpoint of experimental design and analysis, there are many ways A/B testing can go wrong. In this talk, based on my direct personal experience (either of working for tech companies or speaking at length with people who have) at four major players in Silicon Valley and elsewhere (without naming names), I will detail a number of the pitfalls of A/B testing and propose alternatives that may lead to better outcomes.
|
