Eliciting Priors for Bayesian Prediction of Daily Response Propensity in Responsive Survey Design: Historical Data Analysis vs. Literature Review

• Activity Number: 472 - Paradata and Responsive Survey Design
• Type: Contributed
• Date/Time: Thursday, August 6, 2020 : 10:00 AM to 2:00 PM
• Sponsor: Survey Research Methods Section
• Author(s): Brady T West* and Michael R Elliott and James Wagner and Stephanie Coffey and Xinyu Zhang
• Companies: University of Michigan and University of Michigan and University of Michigan and University of Maryland - JPSM and University of Michigan
• Keywords: Responsive Survey Design; Bayesian Analysis; Paradata; Survey Methodology; Data Collection; Response Propensity Modeling

Abstract

Responsive Survey Design (RSD) aims to increase the efficiency of survey data collection via ongoing monitoring of paradata and the introduction of ongoing protocol changes in a manner design to reduce predicted costs and/or reduce predicted survey error. Daily response propensities for all active sampled cases are among the most important parameters for live monitoring of data collection outcomes; thus making sound predictions of these propensities essential for the success of RSD. Because RSD relies on real-time updates of prior beliefs about key design parameters like response propensity, it stands to benefit from Bayesian approaches. However, empirical evidence of the merits of these approaches is lacking in the literature, and elicitation of informative prior distributions is required for their effectiveness. Using a real data collection employing RSD (the National Survey of Family Growth, or NSFG), we evaluate the ability of two prior elicitation approaches to improve predictions of daily response propensity: analyzing historical data from similar surveys and literature review. We find evidence of improved predictions in both cases, especially when using historical data.