Variance Estimation for Volunteer Panel Web Surveys Using Propensity Score Adjustment and Calibration Adjustment

• Activity Number: 445
• Type: Topic Contributed
• Date/Time: Wednesday, August 10, 2005 : 2:00 PM to 3:50 PM
• Sponsor: Section on Survey Research Methods
• Author(s): Sunghee Lee*+ and Richard L. Valliant
• Companies: University of California, Los Angeles and University of Michigan
• Keywords: variance estimation ; web surveys ; calibration adjustment ; propensity score weighting

Abstract

Combination of propensity score adjustment and calibration is shown to reduce bias in volunteer panel Web surveys. This combination first adjusts the design weights correcting for selection bias from nonrandomized sampling through propensity score adjustment and re-adjusts these adjusted weights correcting for coverage bias through calibration. Therefore, the final set of weights is comprised of multiple components, and the estimator no longer takes a linear form. When these adjustments are jointly used, there is no clear approach for deriving variance. This study will compare three variance estimation methods for the adjusted estimates through simulation. The first method resembles what commercial statistical software uses. The second approach uses the variance estimator in Deville and Särndal modified from the asymptotic variance estimator for the generalized regression estimator. The third method will derive variance using jackknife replication method. Variance estimates from these approaches will be compared to the simulation variances, and their coverage rates of the nominal confidence intervals will be examined with the magnitude of bias reduction achieved by the adjustment.