Abstract:
|
Field experiments are the gold standard for making informed decisions and developing scientific knowledge at Internet service companies. Yet in some situations, while data may be abundant, running experiments may not be feasible. We consider the case of estimating peer effects in networks, an area of research largely thought to be riddled with latent confounding. Using a 67 million person field experiment as a gold-standard, we examine the bias of a range of observational estimators -- from a naive observational estimate, to high-dimensional propensity score stratification using thousands of covariates. We find that while naive observational estimators can hugely overstate peer effects, by 320% that of the true effect, high-dimensional methods that rely on closely related prior behaviors can reduce bias to only 30%. These results are cautionary when such variables are not present, and provide motivation for additional validation and use of observational methods.
|
ASA Meetings Department
732 North Washington Street, Alexandria, VA 22314
(703) 684-1221 • meetings@amstat.org
Copyright © American Statistical Association.