|
Activity Number:
|
132
- Statistical Analysis for Networks
|
|
Type:
|
Contributed
|
|
Date/Time:
|
Monday, July 30, 2018 : 8:30 AM to 10:20 AM
|
|
Sponsor:
|
Section on Statistical Learning and Data Science
|
|
Abstract #330492
|
|
|
Title:
|
Designing A/B Tests in a Collaboration Network
|
|
Author(s):
|
Sangho Yoon*
|
|
Companies:
|
Google
|
|
Keywords:
|
A/B test;
network;
connected component;
network effects;
randomization;
stratification
|
|
Abstract:
|
We discuss an approach to the design of experiments in a network. In particular, we describe a method to prevent potential contamination (or inconsistent treatment exposure) of samples due to network effects. We present data from Google Cloud Platform (GCP) as an example of how we use A/B testing when users are connected. Our methodology can be extended to other areas where the network is observed and when avoiding contamination is of primary concern in experiment design. We first describe the unique challenges in designing experiments on developers working on GCP. We then use simulation to show how proper selection of the randomization unit can avoid estimation bias. This simulation is based on the actual user network of GCP.
|
Authors who are presenting talks have a * after their name.
