Monitoring Response Data and Respondent Representativeness to Develop Adaptive Survey Design Interventions

• Activity Number: 361
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Survey Research Methods Section
• Author(s): Stephanie Coffey and Benjamin Reist and Gina Walejko and Allison Zotti*
• Companies: U.S. Census Bureau and U.S. Census Bureau and U.S. Census Bureau and U.S. Census Bureau
• Keywords: Adaptive Design ; Datat Monitoring ; Metrics

Abstract

Adaptive survey design researchers and practitioners tailor data collection features to sample units to maintain data quality while reducing cost or improve data quality within a fixed budget. Examples of treatments include how sample units are contacted, the level of incentive a sample unit receives, or whether further attempts on a unit should be made. To tailor treatments, survey designers use data known about sample units before data collection begins as well as response data and operational paradata collected during data collection.

Adaptive design requires metrics to assess data quality and progress, and reports and data visualizations to monitor those metrics over time. These monitoring tools help determine the best intervention for the next round or phase of the data collection.

This presentation discusses quality and progress metrics in two very different surveys, the Survey of Income and Program Participation (SIPP) and the National Teacher and Principal Survey (NTPS). We discuss metrics to measure representativeness, and progress indicators to inform how respondents interact with the survey, as well as how they could inform data collection interventions.