Effect of Data-Collection Mode on Response Rates and Data Quality in Voting Survey of Active Duty Military

• Activity Number: 34
• Type: Contributed
• Date/Time: Sunday, August 9, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): David McGrath* and Tim Markham and Eric Falk and Kim Hylton and Fawzi Al Nassir
• Companies: DOD/DMDC and Research, Surveys, and Statistics Center and Research, Surveys, and Statistics Center and Research, Surveys, and Statistics Center and Research, Surveys, and Statistics Center
• Keywords: Survey ; Mode ; Phone ; Web ; Military ; Voting

Abstract

In November, 2014, the Defense Research, Surveys, and Statistics Center (RSSC) conducted for the Federal Voting Assistance Program (FVAP) a survey of active duty military members. To improve response rates, RSSC designed a randomized experiment to assign sample members to either of two modes of data collection (web or telephone). Historically, response rates to web surveys of active duty military have had low response rates, and an earlier phone survey produced better response rates, especially for typically difficult to survey young military members. Response rates to the web survey were 10% while the phone produced 20% response rates. The successful integration of the web and phone surveys was critical, and we examined the data to assess mode effects. Unweighted estimates from the phone survey were significantly different from data from the web survey. The determination whether to use the phone data in production estimates hinged on whether we could attribute the mode differences to either improved sample balance (sample composition) from the web survey, or improved measurement.