Data Collection Mode Effects Controlling for Sample Origins in an Internet Panel Survey

• Activity Number: 411
• Type: Contributed
• Date/Time: Thursday, August 12, 2004 : 8:30 AM to 10:20 AM
• Sponsor: Section on Survey Research Methods
• Author(s): J. Michael Dennis*+ and Cindy Chatt
• Companies: Knowledge Networks, Inc. and University of Nebraska
• Keywords: mode effects ; Knowledge Networks ; internet panel ; telephone

Abstract

We evaluate telephone- and internet-based modes of survey data collection by controlling for sample origin. Previous research has focused on sample effects only. Our main result is that substantive response differences are primarily associated with mode of data collection and not with sample origin. Sample origin is controlled by conducting both internet and telephone interviews with members of the Knowledge Networks (KN) web-enabled panel. The survey, which was sponsored by RTI International, measures policy and civic attitudes regarding 9/11 in early 2002, and was designed by RTI International and the Odum Institute at the University of North Carolina. The survey analysis is based on 2,979 web interviews with KN panelists, 300 telephone interviews with KN panelists, and 600 telephone interviews with persons that refused to join the KN panel or else take the web panel survey. The differences caused by mode in this internet vs. telephone study were strikingly similar to the telephone versus mail mode effects found in civic attitude studies by Tarnai and Dillman and in telephone versus face-to-face mode effects by Krysan. These studies found a tendency (which we confirm) for telephone respondents to answer on the extreme positive end of the scale. The internet respondents are more likely than both telephone sample groups to use the full range of scales.