Nonresponse Weight Adjustment in the Census Bureau's Probability and Nonprobability Tracking Surveys

• Activity Number: 549 - Integration of Design and Estimation Approaches in the Use of Auxiliary Data with Sample Surveys
• Type: Topic Contributed
• Date/Time: Thursday, August 11, 2022 : 10:30 AM to 12:20 PM
• Sponsor: Survey Research Methods Section
• Author(s): Eric Slud*
• Companies: US Census Bureau
• Keywords: design-based; calibration; missing data; large-sample theory; large weight movements; benchmarks

Abstract

This talk concerns survey weighting and comparison of two Tracking Surveys, a probability-sampled (random-digit-dialed) telephone survey and an opt-in Web survey, conducted by a contractor for the Census Bureau. Both had very low response rates (< 10%) and raw demographics differing markedly from the general US population. According to well-established theory, when inverse-inclusion-probability survey weights are calibrated (without nonresponse!) to true totals in a probability survey, the design-based estimates are consistent and have reduced variances, and the weights move very little. In the Tracking Surveys, the movement of weights was necessarily large. Methodologically challenging aspects of weighting included missing data in the calibration variables and the lack of theoretical guidance for variance estimation. This setting led us to develop new design-based theory for raked weights when the true population weights satisfy a loglinear model with base-weight offsets, and this theory leads to valid design-based large-sample variance formulas. The take-away point is that weight calibration in surveys with low response or non-probability design is unavoidably model-based.