Online Program Home
My Program

Abstract Details

Activity Number: 502
Type: Contributed
Date/Time: Wednesday, August 3, 2016 : 8:30 AM to 10:20 AM
Sponsor: Survey Research Methods Section
Abstract #318942 View Presentation
Title: Inference from Complex Survey-Embedded Field Experiments
Author(s): Robert Ashmead* and Eric Slud
Companies: U.S. Census Bureau and U.S. Census Bureau
Keywords: survey sampling ; randomized experiments ; design-based estimation ; randomization-based estimation

Experiments embedded in surveys are a valuable way to test potential survey improvements; however, inference from these experiments can be difficult because one must account for the survey design aspects. Operational issues can restrict possible experimental design choices. In this paper, we discuss a design-based and a randomization-based approach to inference for experiments in an embedded survey. Additionally, some surveys are ongoing and present the option of utilizing data from previous time periods as well as the data from the test period. We discuss and compare an estimator that utilizes data only from the test period with one that compares outcome differences between the study period and a prior period across study treatment groups. Using an example from the American Community Survey (ACS) as well as a simulation, we compare the two inferential approaches and estimators. We find that the permutation-based inferential approach is a flexible and robust method for analyzing embedded experiments and that when the correlation is large between time periods, the difference in time-period differences may provide inferences with greater precision.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

Copyright © American Statistical Association