Online Program

Integrating Probability and Nonprobability Sampling Frames To Survey the Muslim American Population

Courtney Kennedy, Abt SRBI 
Scott Keeter, Pew Research Center 
*Mike Battaglia, Abt Associates 
KP Srinath, Abt Associates 
Leah Christian, Pew Research Center 
Gregory Smith, Pew Research Center 
Chintan Turakhia, Abt SRBI 

Keywords: list frames, multiple frames, design tradeoffs, coverage

Because of the proliferation of consumer databases, lists of rare but scientifically noteworthy populations are increasingly common. These can serve as efficient sampling frames because the incidence of the rare population on the list is relatively high. The downside is that they often provide incomplete coverage of the population, and may be biased. Researchers have addressed this dilemma by supplementing a list frame with a less efficient but probability-based frame that provides near complete coverage of the rare population. If de-duplication is possible, the list frame can be treated as a special stratum of the probability-based frame, and facilitate the calculation of full sample probability-based estimates. To reduce screening costs, units in the list stratum are oversampled relative to units in the remainder of the probability frame. We refer to this sampling approach as probability-based with an oversampled list stratum (or “POLS”).

In US telephone surveys, POLS designs may require both landline and cell random digit dial (RDD) frames to achieve adequate coverage for most rare populations of interest. Little empirical work exists to guide researchers on balancing the cost, variance, and bias tradeoffs inherent in POLS designs. We address this research gap by evaluating the performance of a POLS design in a 2011 survey of Muslim Americans, who are less than 1% of the US adult population. The design features a list frame sample, landline and cell RDD samples, and a re-contact sample of Muslim Americans interviewed in prior dual frame RDD surveys. The analysis focuses on design effects (loss in precision) generated by various sampling features (oversampling of the list stratum, overlap in the RDD frames, integration of the re-contact sample) and weighs those against cost savings associated with a POLS design. This study demonstrates how researchers can use POLS designs to generate probability-based estimates for rare populations.