Applying Calibration Weighting to Real-Time Surveys: Some Lessons Learned from the Research and Development Survey (RANDS)

• Activity Number: 105 - Advances in Statistical Methods and Models for Real-Time Surveys
• Type: Topic Contributed
• Date/Time: Monday, August 8, 2022 : 8:30 AM to 10:20 AM
• Sponsor: Survey Research Methods Section
• Author(s): Yulei He* and Katherine Irimata and Van Parsons and Bill Cai and Rebecca Hu and Rong Wei and Guangyu Zhang and Hee-Choon Shin
• Companies: National Center for Health Statistics and National Center for Health Statistics and CDC and CDC and CDC and CDC and CDC and CDC
• Keywords: Calibration; Raking; Propensity Score; Web Survey; Standardized Bias; Variable Selection

Abstract

Research and Development Surveys (RANDS), conducted by National Center for Health Statistics (NCHS), is a series of primarily online surveys based on commercial probability panels. RANDS data and estimates can be used to supplement those from traditional household surveys such as the National Health Interview Survey (NHIS). However, to ensure and possibly improve their quality, we calibrated weights from the original RANDS data using information from benchmark surveys such as NHIS. This talk summarizes some of the relevant experiences and findings. For example, our empirical research has demonstrated that after the calibration, RANDS estimates are overall closer to those from the benchmark survey. In addition, the selection of calibration variables plays a key role in determining the effectiveness of calibration, and calibration effectiveness varies across outcomes of interest and important subgroups.