Data Collection Mode Effects: Empirical Comparisons Between CATI-Only vs. Mixed Mode in the 2003 National Survey of Recent College Graduates

• Activity Number: 420
• Type: Contributed
• Date/Time: Wednesday, August 5, 2009 : 8:30 AM to 10:20 AM
• Sponsor: Section on Survey Research Methods
• Author(s): Zhanyun Zhao*+ and Donsig Jang and Kelly H. Kang
• Companies: Mathematica Policy Research, Inc. and Mathematica Policy Research, Inc. and National Science Foundation
• Keywords: NSRCG ; Data collection mode ; random sample ; response rate ; survey estimates

Abstract

In 2003, the National Survey of Recent College Graduates (NSRCG) data collection protocol was changed from interviewer administration (CATI) to multi-mode administration (mail, Web, and CATI). To assess the effects of the data collection protocol change on the changes of estimates over time, an experiment was designed in advance before the 2003 fielding began. The 2003 sample was randomly partitioned into two groups: a control sample of 2,000 graduates to whom the survey was administered via CATI, and a treatment sample of 16,000 graduates to whom the multi-mode survey was administered. Comparisons were made between two groups on unit and item response rates, item editing rates, and key survey estimates. Statistically significant differences will point to a need for further analyses to assess the impact of the multi-mode data collection protocol on the NSRCG time-series analyses.