A Representative Sampling Method for Causal Inference in Social Network Experiments

• Activity Number: 439 - Topics in Marketing
• Type: Contributed
• Date/Time: Thursday, August 12, 2021 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistics in Marketing
• Author(s): Yanyan Li and Qing Liu* and Sha Yang
• Companies: University of Southern California and University of Wisconsin-Madison and University of Southern California
• Keywords: RCT; Causal Inference; A/B testing; Network Experiments; Random Sampling; Representative Sampling

Abstract

Random Control Trials (RCTs) have been widely implemented on social networks to test various effects ranging from product design to marketing effectiveness. A standard practice in such network-based RCTs is to randomly sample users into treatment and control conditions, and estimate the average treatment effect by directly comparing outcomes of the two samples. In this research, we first demonstrate that such random sampling approach can lead to biased causal inference due to different network characteristics observed between the treatment and the control samples. More importantly, we show that such biases cannot be corrected even after controlling for the observed network properties in the estimation. We propose an efficient Metropolis-Hastings algorithm to draw representative samples for both the treatment and control units, such that the treatment and the control samples are comparable in their network properties (e.g. degree distribution) and both preserve the population network properties. Through simulations and counterfactual analysis, we demonstrate the benefits of our representative sampling over the standard random sampling methods and discuss managerial implications.