Measures of the Degree of Departure from Ignorable Sample Selection

• Activity Number: 442 - State-Of-The-Art Inferential Approaches for Non-Probability Samples
• Type: Invited
• Date/Time: Wednesday, July 31, 2019 : 8:30 AM to 10:20 AM
• Sponsor: Survey Research Methods Section
• Author(s): Phil Boonstra* and Brady T. West and Roderick J Little and Rebecca Andridge
• Companies: University of Michigan and University of Michigan and University of Michigan School of Public Health and The Ohio State University College of Public Health
• Keywords: selection bias; pattern-mixture model; nonresponse; auxiliary proxy; standardized bias

Abstract

Non-probability samples, e.g. most internet-based surveys, are by definition subject to a non-ignorable selection mechanism and thus prone to yield biased inferences when analyzed with methods suitable only for true probability samples. Nonetheless, their convenience and easy availability make such data difficult to simply disregard. With this tension in mind, we propose a standardized measure of unadjusted bias (SMUB), which estimates bias in a continuous outcome of interest as a function of the degree of non-random sampling. The measure is derived from the theory of normal pattern-mixture models, and its calculation therefore only requires knowledge about an administrative proxy for the outcome in the entire population. Extending these bias measures to the case of binary outcomes is also possible. In all cases, the degree of non-random sampling can be used as a sensitivity parameter to quantify potential bias in a given non-probability sample. We illustrate these measures with a simulation studies and well as with data from the National Survey of Family Growth, where certain subsets, e.g. smartphone users, can be viewed as non-probability samples from the entire population.