Thursday, November 10
Data Quality and Measurement Error
Thu, Nov 10, 1:30 PM - 2:55 PM
Hibiscus B
Exploring the Interaction Between Question Design and Interviewer Behavior on Measurement Error

Expanded and Nonconforming Answers in Standardized Survey Interviews (303386)

Tom Koole, University of Groningen 
Yfke Pieternel Ongena, University of Groningen 
*Sanne Unger, Lynn University 

Keywords: expansions, probe, nonconforming answers

Respondents in standardized survey interviews do not always answer closed-ended questions with just a type-conforming answer, such as "yes" or “three.” Instead, they sometimes do extra work by expanding the type-conforming answer, or by providing a response that does not contain a type-conforming answer at all. Standardized survey methodology aims to avoid such answers because they are found to be the most important cause for survey interviewers to deviate from their script. Such deviations can introduce interviewer error and threaten data quality.

A conversation analytic study of survey interviews shows that expanded and nonconforming answers can be divided into six types: serial extras, uncertainty markers, pre-expansions, post-expansions, nonproblematic nonconforming answers, and problematic nonconforming answers. Our analysis shows that respondents employ each of these six types of answers and expansions to target a specific aspect of the interview situation. Expanded responses indicate imprecision or similarity to previous answers, change the terms of the question, or adjust the scope of the answer. Nonconforming responses are used as reportings, and invite the interviewer to select the correct answer option.

In this presentation, we show that expanded and nonconforming responses do not lead to intervention by the interviewer and are treated as unproblematic. The only problematic answer expansion type is the nonconforming answer that is designed as dispreferred by the respondent. A follow-up quantitative analysis of 610 Computer-Assisted Telephone Interviews (CATI) shows that nonproblematic expansions are overwhelmingly accepted by interviewers, while nonconforming answers are in most cases followed by interviewer probing. Problematic nonconforming answers have a significantly higher rejection rate than all other expanded and nonconforming answers.