Alternative Indicators for the Risk of Nonresponse Bias: A Simulation Study

• Activity Number: 141
• Type: Contributed
• Date/Time: Monday, July 30, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Section on Survey Research Methods
• Author(s): Raphael Nishimura*+ and James Wagner and Michael R. Elliott
• Companies: University of Michigan and University of Michigan and University of Michigan
• Keywords: Missing data ; Nonresponse bias ; Nonresponse indicators ; Survey data quality measures

Abstract

The growth of nonresponse rates for social science surveys has led to increased concern about the risk of nonresponse bias. Unfortunately, the nonresponse rate is a poor indicator of when nonresponse bias is likely to occur. We consider a set of alternative indicators - including the Fraction of Missing Information, R-Indicators, the coefficient of variation of subgroup response rates, and model fit statistics such as R-squared, pseudo R-squared, and the area under an ROC curve. A simulation study is used to explore how each of these indicators performs under a variety of circumstances. The simulations vary the missing data mechanism (MCAR, MAR, and NMAR), the strength of covariates in predicting response and survey outcome variables, and the impact of the misspecification of models. Finally, we will discuss how these indicators can be used when creating a plausible account of the risks