A Method to Assess Nonresponse Bias During Fieldwork

• Activity Number: 250
• Type: Contributed
• Date/Time: Monday, August 10, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Peter Smith* and Solange Correa and Gabriele Durrant
• Companies: University of Southampton and University of Southampton and University of Southampton
• Keywords: call record data ; household longitudinal study ; nonresponse bias monitoring ; paradata ; survey quality indicator

Abstract

A method to monitor survey outcomes during fieldwork is proposed. It assesses nonresponse bias using call record data by comparing estimated and true distributions of specific survey variables at each call attempt using dissimilarity indices. These indices are compared with other indicators, such as response rate, R-indicators and coefficients of variation. Results for data from waves 1 and 2 of the UK Household Longitudinal Study show that survey estimates tend to stabilise after around 5 call attempts and, in some cases, at levels that depart significantly from the corresponding true values even after high response rates are achieved. The study demonstrates that most indicators in current use, although adequate to assess nonresponse bias after data collection, are not effective in capturing nonresponse bias during the call process. The study concludes that dissimilarity indices and coefficients of variation have the best properties. This research has direct implications to responsive and adaptive survey designs. Focusing on simply achieving high response rates is not cost-effective and considering alternative indicators is crucial to guide real-time data collection strategies.