Contrasting Stylized Questions of Sleep with Diary Measures from the American Time Use Survey (303129)

Keywords: questionnaire evaluation, stylized questions, time use, diary, data quality

Data on how people spend their time is of great interest to researchers, government, and health organizations to better understand economic and societal trends. In the American Time Use Survey (ATUS), interviewers use a set of scripted, open-ended questions to walk respondents chronologically through their activities during the prior 24-hours. In contrast, other surveys ask people about “the average, normal, or typical” time spent on activities, or stylized questions. Estimates of sleep duration vary between diary and stylized questions. In 2013, the ATUS reported that U.S. adults spend an average of 8.6 hours per day sleeping. But stylized questions, such as those used in the National Health Interview Survey (NHIS), find that U.S. adults get an average of 6.9 hours of sleep per day. Our research draws on a variety of evaluation methods to examine reasons for the discrepancy between diary and stylized sleep measures. First, we used behavior coding data of ATUS interview transcripts to find patterns in interviewer and respondent behaviors surrounding questions about sleep and possible sources of measurement error in the ATUS. Cognitive interviews were conducted to better understand the response process for both diary and stylized measures, such as comprehension, retrieval/estimation strategies, and social desirability concerns in reporting on sleep. An experimental study was conducted to investigate the generalizability of the qualitative data and differences in reported sleep duration across diary and stylized measures. Finally, we carried out a validation study where a device tracked participants’ sleep duration over a one-week period. We compare device-measured sleep duration to participants’ reported sleep, using diary and stylized measures, to determine direction of bias. We discuss the strengths and weaknesses of each method and how they can build off one another in the questionnaire evaluation process to gain a deeper understanding of a substantive issue.