Helping Respondents to Format Their Answers: a Question Wording Experiment in a Telephone Survey (303104)

Keywords: survey interviewing, questionnaire design, computer-assisted interviewing, question-answer sequence, interaction analysis, satisficing theory, conversation analysis

In telephone interviews, respondents frequently deviate from prescribed response scales (e.g., instead of choosing one option from a 5-point Likert-type agree-disagree scale, they answer simply ‘yes’ or ‘no’). These deviations yield uncodable answers, called mismatch answers, which may occur in as much as 65 percent of the administrations of a single question. Although standardization rules require interviewers to repeat all alternatives, conversational principles will encourage interviewers to show they are active listeners and to not ignore the respondent’s attempt of providing an adequate answer. In an experiment within the Nebraska Annual Social Indicators Survey, we tested our hypothesis that question wording may aid respondents in formatting answers adequately. Questions were manipulated on the explicitness of the response task, and the type of response alternatives. The results show that explicitness of the task did not reduce mismatch answers, but it did reduce the number of reports (i.e., relevant information from which an answer may be derived). Using common-language response alternatives did significantly reduce the number of mismatch answers, replicating Ongena and Dijkstra’s (2010) findings. We complement the findings with the qualitative methodology of conversation analysis (CA). We use CA to see what respondents do by providing mismatch answers. For instance, a distinction can be made between colloquial answers (i.e., respondents do not repeat the exact wording of response alternatives), tentative answers (indicating uncertainty or imprecision), and answers that seem to be over-precise, in which information is added to the response alternative. By examining interactions in detail, CA provides an opportunity to suggest possible reasons for formatting problems.