29 – SPEED: Survey Methods, Transportation Studies, SocioEconomics, and General Statistical Methods Part 1
How Hard Is It to Remove Mode Effects in Multimode Surveys? Basic Weighting V. Three Model-Based Methods
Thomas Brassell
ICF
James Dayton
ICF
Stephen Immerwahr
NYC Department of Health and Mental Hygiene
Matt Jans
ICF
Amber Levanon
ICF
Sahnah Lim
ICF
Kelly Martin
ICF
Randy ZuWallack
ICF
Multimode data collection is increasingly popular as survey funders and data collectors try to counteract declining response rates and increasing nonresponse bias risk. Using multiple modes can capture respondents who are different from those responding by the primary mode, thus reducing nonresponse bias. However, for long-running single-mode surveys, this can introduce undesirable changes in trends. This presentation compares three mode adjustment methods (Kolenikov and Kennedy, 2014) with standard weighting using the 2017 New York City Social Determinants of Health survey. Three health outcomes that differed between random digit dial (RDD) and address-based samples (ABS) in unweighted analyses were the focus. Standard weighting removed mode differences for all three outcomes. Further adjustment with a regression method (RM), multiple imputation (MI) method, and an implied utility multiple imputation (IUMI) method moved estimates closer to weighted RDD estimates, which was considered the gold standard. These adjustments ranged from -1.41 to 1.35 percentage points beyond basic weighting. Results are discussed in the context of mode transition and implementation.