Online Program Home
My Program

Abstract Details

Activity Number: 472 - Statistical Methods for Causal Inference
Type: Contributed
Date/Time: Wednesday, July 31, 2019 : 8:30 AM to 10:20 AM
Sponsor: Section on Statistics in Epidemiology
Abstract #307051 Presentation 1 Presentation 2
Title: An Evaluation of Model-Based and Design-Based Variance Estimators in Completely Randomized Experiments
Author(s): Stanley Lubanski* and Peter Steiner
Companies: University of Wisconsin-Madison and University of Wisconsin
Keywords: variance estimation; randomized experiment; regression adjustment; average treament effect; heteroscedasticity; experimental design

Recently, design-based variance estimators have been proposed to accompany estimates of average treatment effects in completely randomized experiments. While justified by theory, how much of a difference might these design-based variance estimators make in practice when compared to more conventional, model-based variance estimators? This paper seeks to answer this question using simulations to compare the variance estimators (two design-based estimators from Schochet (2015), the conventional regression estimator, and the heteroscedasticity-consistent estimator, HC3) while varying several variance-related factors, including: the scope of inference, outcome heteroscedasticity, balanced/unbalanced design, the presence or absence of covariate adjustment, and sample size. The findings: 1) the design-based estimators are best when heteroscedasticity is caused by idiosyncratic noise and the design is balanced; else they perform similarly to the HC3 estimator and 2) design-based estimators may benefit from a variance-inflation-factor correction.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program