|
Abstract:
|
Randomized controlled trials play an important role in how internet companies predict the impact of policy decisions and product changes. Heterogeneity in treatment effects refers to the fact that, in such digital experiments, different units (people, devices, products) respond differently to the applied treatment. This article presents a fast and scalable Bayesian nonparametric analysis of heterogeneity and its measurement in relation to observable covariates. New results are provided for quantifying uncertainty about four different statistics that practitioners use to index heterogeneity: two based on linear projections and two based upon regression tree partitioning. Throughout, the work is illustrated with an example experiment involving 21 million unique users of EBay.com.
|
