Online Program Home
My Program

Abstract Details

Activity Number: 292
Type: Invited
Date/Time: Tuesday, August 2, 2016 : 8:30 AM to 10:20 AM
Sponsor: Section on Physical and Engineering Sciences
Abstract #318295
Title: Statistical Design for Online Experiments Across Desktops, Tablets, Smartphones (and Maybe Wearable Gadgets)
Author(s): Peter Qian* and Soheil Sadeghi and Neeraj Arora
Companies: University of Wisconsin - Madison and University of Wisconsin - Madison and University of Wisconsin - Madison
Keywords: Design of experiments ; Online Experiments ; A/B Testing ; Marketing ; E-Marketing ; Factorial Designs

A/B testing and multivariate testing have become the bread-and-butter for applications as varied as website optimization, mobile apps and email campaigns. An emerging problem in this area is how to conduct online experiments simultaneously across multiple platforms such as desktops, tablets, smartphones (and maybe wearable gadgets). A simple way is to run tests for different platforms separately. An obvious drawback of this approach is that it does not borrow information from all the sources. To overcome this limitation, we propose a new experimental design framework that allows for optimizing testing for different platforms but also enabling comparing and pooling data across the platforms. Based on a new criterion called sliced effect hierarchy, this framework introduces a "sliced design" idea to partition test units into homogenous slices. The different slices are then used to optimize testing for each platform and the combined design makes it possible to benchmark and correlate testing performance of the different platforms. We demonstrate the effectiveness of the proposed method by using email design application for an online magazine.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

Copyright © American Statistical Association