Estimating Mode Effects Without Bias: A Randomized Experiment to Compare Mode Effects Between Face-to-Face Interviews and Web Surveys

• Activity Number: 662
• Type: Invited
• Date/Time: Thursday, August 8, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Survey Research Methods Section
• Author(s): Douglas Rivers*+ and Lynn Vavreck
• Companies: Stanford University and UCLA
• Keywords: Total survey error ; Interview mode ; Web survey ; Nonresponse

Abstract

Because survey modes usually involve different sampling designs, estimates of mode effects are often confounded with differences in sample composition. We conducted an experiment where 1,000 subjects were randomized between two interview modes: a self-administered Web survey and the identical survey conducted by an interviewer. The randomization allows unbiased estimation of mode effects. Using a latent class model, we show that interviewers tend to elicit less extreme responses on attitudinal scales and lower correlations among items than in the self-administered mode. These differences cannot be attributed to sample composition and explain some anomalous results found in earlier studies.