Impacts of Question Format on Prevalence Estimates, Item Nonresponse, and Response Times: A Split-Ballot Experiment of Disability Questions on the National Health Interview Survey

• Activity Number: 349
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Survey Research Methods
• Author(s): Julie Dawson Weeks*+ and James Dahlhamer
• Companies: National Center for Health Statistics and National Center for Health Statistics
• Keywords: questionnaire design ; experimental design ; predictive validity ; data quality ; response times ; disability

Abstract

In 2009, the National Health Interview Survey (NHIS) conducted a split-ballot experiment using the six disability questions often referred to as the ACS disability set. The questions were asked at the end of the family interview, and families were randomly assigned to one of two question formats: family-style (e.g., "Is anyone deaf or does anyone have serious difficulty hearing?" If "yes," "Who is it?"); and person-style (e.g., "Is < person> deaf or does < person> have serious difficulty hearing?" The question is repeated for each eligible person in the family).

In this paper, we explore differences in disability prevalence estimates by question format, along with direct and indirect indicators of data quality such as item nonresponse, response consistency, and response times. We further explore correlational or predictive validity of disability measures by question format. That is, which measures, those administered family-style or person-style, correlate more strongly with variables known to be highly associated with disability status? We discuss the implications of our results for the larger field of survey design and error, as well as an upcoming content redesign of the NHIS