The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.
Abstract Details
|
Activity Number:
|
41
|
|
Type:
|
Contributed
|
|
Date/Time:
|
Sunday, July 31, 2011 : 2:00 PM to 3:50 PM
|
|
Sponsor:
|
Section on Statistics and Marketing
|
| Abstract - #302913 |
|
Title:
|
Choice of the Randomization Unit in Online Controlled Experiment
|
|
Author(s):
|
Shaojie Deng*+ and Roger Longbotham and Toby Walker and Ya Xu
|
|
Companies:
|
Microsoft Corporation and Microsoft Corporation and Microsoft Corporation and Microsoft Corporation
|
|
Address:
|
One Microsoft Way, Redmon, WA, 98052,
|
|
Keywords:
|
controlled experiment ;
AB testing ;
online experiment ;
randomization test
|
|
Abstract:
|
Controlled experiment has been used widely to support data driven decision making for on-line businesses. By applying appropriate randomization of the experiment units, causal inference can be established. The choice of the experiment unit for randomization can vary. User and page view are two mostly used units. Moreover, the analysis unit is sometimes different from the experiment unit. There are pros and cons in choosing which experiment unit to use and the choice affects the downstream statistical analysis. Generally for page level metrics, randomization by page will have an edge in power due to variance reduction. In this paper, we compare the two experiment units and provide a method to correctly analyze a page view randomization experiment in a two layer randomization framework.
|
The address information is for the authors that have a + after their name.
Authors who are presenting talks have a * after their name.
Back to the full JSM 2011 program
|
2011 JSM Online Program Home
For information, contact jsm@amstat.org or phone (888) 231-3473.
If you have questions about the Continuing Education program, please contact the Education Department.