Data Quality in the National Health Interview Survey: Examining the Joint Effects of Question, Respondent, and Interviewer Characteristics (303149)

Keywords: questionnaire design, measurement error, paradata

A variety of factors and design characteristics have been posited to influence the quality of survey responses. At the item level, question type, format, and length have been linked to measurement error, as has the mode or method of data collection. Respondent characteristics such as age, race/ethnicity, gender, and education---along with topical knowledge, interest, and motivation---have also been found to influence the quality of survey responses. Finally, interviewer demographics, skills, and expectations can impact the quality of elicited responses.

Although the survey methods field is replete with research on measurement error and data quality, the bulk of this work tends to focus on just one of the aforementioned sources of error. More recently, however, researchers have begun to examine the joint effects of question, respondent, and interviewer characteristics on data quality.

In this paper, we expand on prior work by examining the joint effects of question, respondent, and interviewer characteristics on data quality using the 2014 National Health Interview Survey (NHIS), a large, nationally representative household health survey. We develop a hierarchical data set with questions nested within respondents nested within interviewers. We then use multilevel modeling to estimate the contribution of each level to variability in item nonresponse and response times.

We add to the literature in three important ways. First, we utilize two measures of data quality: response times and item nonresponse. Second, we incorporate a broader range of question characteristics. And third, we include a set of paradata variables tapping concerns and behaviors of household members. We conclude with a discussion of the varying contributions of questions, respondents, and interviewers to data quality and the implications of our findings for questionnaire design and evaluation.