Thursday, November 10
General
Thu, Nov 10, 10:00 AM - 10:40 AM
Promenade Upper
Thursday Poster Session, Part 1

Evaluating Paradata Variables as Measures of Survey Quality (303577)

*Renee Ellis, U.S. Census Bureau 
Thomas Andy Welton, U.S. Census Bureau 

Keywords: data quality, paradata

The purpose of this paper is to evaluate the usefulness of paradata items collected in the Survey of Income and Program Participation (SIPP) to evaluate the quality of survey responses. Researchers study paradata, or information collected during a survey that does not include answers to questions, in order to improve the quality of the collection and design of survey instruments. Paradata has the potential to help us understand data quality. However, evaluating paradata items in regards to how well they relate to the quality of information collected can be challenging because we often lack “ground truth” to use in comparison. Without ground truth, research has found other elements such as time spent on questions, don’t knows/refuses, or missing/incomplete data impact data quality. SIPP is a longitudinal, nationally representative survey that collects information about program participation, jobs, and income. In wave 1 of the 2014 SIPP, a flag based on administrative records was created to adjust for errors in a question about the receipt of Social Security Insurance (SSI). This gives us a measure of truth for those who answered it. We use this flag in addition to total time spent on the survey and missing/incomplete data as quality measures. We then compare additional paradata measures using bivariate comparisons and regression modeling in order to evaluate their utility in determining quality. The potential paradata measures include: timing of individual sections of the survey, interviewer access of definitions/help files, presence of survey administration errors, receipt of permission by the interviewer to record the interview, respondent use of records in answering asset questions, and rounding of select values in income and asset sections of the interviews.