An Exploration of How Survey Questions Asking About Behavior Prompt Respondents to Categorize Themselves (303325)

Keywords: Cognitive interviewing, Question design

Many survey questions ask respondents to report on their behavior. This paper will demonstrate that fact-based behavior questions are sometimes not interpreted as such by respondents. Instead of reporting actual behavior, respondents sometimes provides answers based on self-perception. The type of survey questions most likely to invoke this type of response pattern are those that implicitly dichotomize respondents into categories relevant to the topic (for example, smokers vs. nonsmokers). Data are examined from reports in Q-Bank, an on-line library of cognitive interviewing final reports. Survey topics include smoking, cancer control, dental health, diabetes, and alternative medicine. Through an examination of findings from these studies, the paper will identify common interpretive patterns among these disparate health topics, summarize the connective theme among the patterns, and posit a general question design principle that is grounded in empirical evidence. Results will show how the categorization of respondents by survey questions can potentially create problems in the response process. Questions that perform as “identity questions” can be more difficult to answer and lead to response error because respondents must first make a decision regarding their self-perception in order to answer the question. When respondents cannot place themselves in the categories assumed by a question, they struggle in choosing a response option, decline to answer, or interpret the question differently than intended. This paper will also examine potential solutions and provide recommendations to survey designers.