423 – Contributed Poster Presentations: Survey Research Methods Section
Evaluating Estimation Methods for Combining Probability and Nonprobability Samples Through a Simulation Study
Nada Ganesh
NORC at the University of Chicago
Edward Mulrow
NORC at the University of Chicago
Vicki J. Pineau
NORC at University of Chicago
Michael Yang
NORC at the University of Chicago
There is growing demand for survey estimation methods to combine probability and nonprobability samples for improved cost efficiency and more timely data dissemination. Researchers have proposed a range of estimation methods involving nonprobability samples, four of which are: (1) Calibration—calibrate total estimates to known population totals; (2) Superpopulation Modeling—use a superpopulation model to derive estimates; (3) Propensity Weighting—model the propensity of inclusion in a nonprobability sample to derive a pseudo weight; and (4) Small Area Modeling—a small area estimation approach developed at NORC (Ganesh et al., 2017). We have previously evaluated these methods empirically using data from a food allergy survey and an NORC internal AmeriSpeak® study. Results showed that the different estimation approaches produce different pseudo weights but largely comparable point estimates (Yang et al., 2018). This paper expands our earlier study by presenting comparative evaluations of these methods through a Monte Carlo simulation study. The simulation data was generated to mimic the coverage bias exhibited by opt-in online panel samples in terms of key characteristics. We compare the properties of composite estimates derived from using different estimation approaches for combining probability and nonprobability samples.