Online Program Home
My Program

Abstract Details

Activity Number: 132 - Statistical Analysis for Networks
Type: Contributed
Date/Time: Monday, July 30, 2018 : 8:30 AM to 10:20 AM
Sponsor: Section on Statistical Learning and Data Science
Abstract #330492
Title: Designing A/B Tests in a Collaboration Network
Author(s): Sangho Yoon*
Companies: Google
Keywords: A/B test; network; connected component; network effects; randomization; stratification

We discuss an approach to the design of experiments in a network. In particular, we describe a method to prevent potential contamination (or inconsistent treatment exposure) of samples due to network effects. We present data from Google Cloud Platform (GCP) as an example of how we use A/B testing when users are connected. Our methodology can be extended to other areas where the network is observed and when avoiding contamination is of primary concern in experiment design. We first describe the unique challenges in designing experiments on developers working on GCP. We then use simulation to show how proper selection of the randomization unit can avoid estimation bias. This simulation is based on the actual user network of GCP.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program