On the Variability of Estimates Based on Propensity-score-weighted Data from Web Panels

• Activity Number: 149
• Type: Topic Contributed
• Date/Time: Monday, August 9, 2004 : 2:00 PM to 3:50 PM
• Sponsor: Section on Survey Research Methods
• Author(s): Annica Isaksson*+ and Gosta Forsman and Stig Danielsson
• Companies: Linkoping University and Linkoping University and Linkoping University
• Keywords: political opinion poll ; variance estimation ; propensity score adjustment

Abstract

This paper deals with variance estimation in web surveys of the general population. Such surveys often utilize a panel of web users, from which samples are selected for various surveys. Inference commonly suffers from considerable problems, including severe selection biases due to low internet penetration in the population and large nonresponse. Thus, good weighting procedures are badly needed. We restrict our attention here to an application of the "propensity-score-weighting" procedure, in which a parallel telephone survey is used to estimate the propensities of being in the web sample. The resulting weights may potentially reduce both selection bias and bias due to nonresponse. It is not obvious, however, how the estimator's variance should be estimated, and estimates are typically presented without adhesive uncertainty measures. This unsatisfying situation is the starting-point of our work. Since textbook variance formulae do not apply on the propensity score estimator, we try instead a model approach. From data collected for an opinion poll before the Swedish 2003 EURO referendum, model-based variance estimates are calculated, and compared with bootstrap estimates.