A Model-Based Approach for Achieving a Representative Sample

• Activity Number: 522
• Type: Topic Contributed
• Date/Time: Wednesday, August 12, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Survey Research Methods Section
• Author(s): George Terhanian* and John Bremer and Jonathan Olmsted
• Companies: NPD Group and Toluna and NPD Group
• Keywords: statistical matching ; optimization ; non-probability sample

Abstract

The enterprise of online research with non-probability samples continues to grow. Still, concerns about its trustworthiness linger. For this reason, researchers have attempted to develop models that replicate probability sampling's positive effects. Such efforts have met with mixed success. To complicate matters, some of the more promising ones have depended on small sample sizes, a small number of samples, or both. Accordingly, there is little consensus on what variables to include in sampling or weighting models when the aim is to minimize bias. Here, we rely on seventeen samples of one thousand, provided by seventeen agencies. We then apply search-based optimization methods to these data to identify the best combination of variables to include in sampling or weighting models. The resulting evidence suggests these models, compared to standard ones, can reduce bias by a third. This learning may be important for science, practice, and policy.