Online Program

Identifying and resolving reporting errors in an annual establishment survey; Lessons from the NSF-NIH Survey of Graduate Students and Postdoctorates in Science and Engineering

Keywords: measurement error, quality control, continuous improvement, post-submission review, respondent burden, systems

Reviewing respondent data for possible reporting errors and recontacting respondents for clarification or correction is a common method for improving data quality. This post-submission review is especially effective for annual data collections where prior responses are good predictors of future responses. This paper describes the methods used to identify reporting errors in the 2007-2010 Survey of Graduate Students and Postdoctorates in Science and Engineering (GSS), cosponsored by the National Science Foundation (NSF) and the National Institutes of Health.

As a census survey that has achieved high response rates throughout its existence, sampling error and nonresponse error are minimized in the GSS. While the NSF is pursuing several efforts to assess and improve survey coverage, this paper details on the development of a quality assurance and quality control system designed to minimize respondent burden, yet improve data accuracy by focusing efforts on critical reporting (measurement) errors. Focus given to process improvements, levels of review, systems integration, status tracking, post-review analyses, and lessons learned from 4 years of post-submission data review.