141 – Tree-Based Methods for Missing Data and Evaluation of Missingness Mechanisms
Analysis of Nonresponse Bias in the 2010 Post-Election Voting Survey of Uniformed Service Members
Tim Markham
Department of Defense
David McGrath
In an era of declining response rates, survey organizations are increasingly examining the effects of nonresponse bias on their survey estimates. As fewer people respond to surveys, the nonresponse bias increases if survey respondents and non-respondents are systematically different from each other, even after the application of sophisticated weighting methods to account for survey nonresponse. To assess the prevalence of this bias in estimates from the Federal Voting Assistance Program’s 2010 Post-Election Voting Survey of Uniformed Service Members, the Department of Defense Manpower Data Center (DMDC) performed a non-response bias study by surveying a random subsample of survey non-respondents. Because the nonresponse sample members were interviewed by telephone and the production survey was a self-administered, Web instrument, DMDC also selected a sub-sample of the production sample for phone interviews to test for mode effects. Results show that modest mode effects exist for some variables, and DMDC uses this result while interpreting the nonresponse study. Analyses of the nonresponse survey reveal that unweighted estimates show large nonresponse bias, but DMDC’s weighting methods effectively reduce, but may not eliminate, this bias in production estimates. From this study, DMDC compared unweighted data from the production and nonresponse surveys with weighted production data; created estimates with the same weighting process as if the survey had closed two weeks earlier or if respondents to the nonresponse study had responded to the production survey to determine the effect of late responders; and created composite estimates comprised of the production data and non-respondent data.
