Does Adding 'Your Best Estimate Is Fine' Affect Data Quality? (303598)

Keywords: data quality, approximation, rounding, heaping, questionnaire design, open-ended questions, respondent burden

Exact answers to open-ended numerical questions (such as "How many times did you visit the grocery store in the last six months?") are sometimes difficult or impossible for respondents to produce. To ease the cognitive burden of these questions, researchers often invite respondents to approximate their answer by adding phrases like "Your best estimate is fine" to the question stem. However, the data quality implications of encouraging estimation are relatively unknown. Additionally, it is not clear whether this implied reduction in cognitive burden is differentially beneficial in self-administered versus interviewer-administered surveys. In this paper, we explore the effect of adding an invitation to estimate to open-ended numerical questions in both a mail survey (the National Health, Wellbeing, and Perspectives Study; n=1,002, AAPOR RR1=17.8%) and a telephone survey (Work and Leisure Today II Survey; n=911, AAPOR RR3=7.8%). We compare rounding behaviors, unweighted survey estimates, question-level response variance, inadequate answers, item nonresponse, and straightlining between questions that have experimentally varied the statement "Your best estimate is fine." These behaviors are examined across several different question topics, such as income and autobiographical recall questions with varying reference periods. In the mail mode, preliminary analyses of 22 questions with this experimental variation show very few differences in data quality. This paper will also include comparison analyses from telephone surveys. We will discuss implications for survey practice and questionnaire design in both telephone and mail surveys.