Balancing Timeliness, Data Quality, and Cost by Optimizing Data-Collection Strategies

• Activity Number: 303
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Survey Research Methods Section
• Author(s): Wan-Ying Chang*+ and Lynn Milan and Steven Proudfoot
• Companies: NSF and NSF/NCSES and NSF/NCSES
• Keywords: timeliness ; survey efficiency ; survey cost ; data quality indicators

Abstract

For large-scale national surveys, gains in cost reduction and increased timeliness must be weighed against the quality of the data produced. Recent developments in adaptive design and data monitoring indicators offer promising tools to assess the tradeoff between quality, cost, and timely data production. The Survey of Doctorate Recipients (SDR), a biennial longitudinal survey of U.S.-trained doctorates in science, engineering, and health, used to collect data for more than 40 weeks in order to achieve a high response rate. In its 2013 survey cycle, the SDR shortened its data collection period by 40 percent. The compressed data collection schedule was implemented by reducing the timing between follow-up contacts while maintaining all of the past contact strategies. We analyzed and compared the weekly progress of the 2010 SDR with the 2013 SDR with respect to representativeness of the response set, non-response bias, data quality at the item level, and level of bias and precision in estimating key survey outcomes. The cost and efficiency of data collection were also tracked and compared over time to yield information on optimizing the design of data collection strategies.