Methods for Identifying Careless Responders in Online Survey Data

• Activity Number: 423 - Contributed Poster Presentations: Survey Research Methods Section
• Type: Contributed
• Date/Time: Tuesday, July 30, 2019 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Elise Amioka* and Joshua Curtis Black and Alyssa Forber and Karilynn Rockhill
• Companies: Rocky Mountain Poison and Drug Center and Rocky Mountain Poison and Drug Center and Rocky Mountain Poison and Drug Center and Rocky Mountain Poison and Drug Center
• Keywords: careless responding; survey validity; bias reduction; exclusion criteria; drug abuse

Abstract

The Survey of Non-Medical Use of Prescription Drugs Program (NMURx) is a system that collects information on drug use and abuse using an online survey instrument. Validity of responses is often a concern for anonymous online surveys, and NMURx is further challenged by collecting information on rare outcomes related to behaviors that may encounter social stigmas. To improve data quality and key outcome estimates, criteria that excluded 0.5% of all respondents for providing careless responses were applied to NMURx data collected in 2018. After consideration of several literature-based and data-driven methods, criteria were based on time spent on demographic questions, response pattern indices, even-odd consistency, and outlier analysis of the total number of drug products endorsed. In certain low volume drug groups, over 20% of endorsements came from careless respondents. 47.1% of the careless respondents provided contradictory answers, and the median time spent on drug use questions was significantly lower among careless respondents (p < 0.0001). These internally consistent results indicate that the criteria identified diverse behaviors in respondents that should be removed.