Impact of Survey Administation Mode on Educational Surveys

• Activity Number: 423 - Contributed Poster Presentations: Survey Research Methods Section
• Type: Contributed
• Date/Time: Tuesday, July 30, 2019 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Yue Jia*
• Companies: Educational Testing Service
• Keywords: Educational Survey; Survey Administration Mode ; NAEP; random group design; Cross mode comparisons

Abstract

This paper investigated the effects of survey administration mode (paper and pencil vs. computer) on the responses of educational surveys. Educational surveys are used to monitor student populations’ academic progress over time. Such surveys draw probability samples from the target population and administer to the sampled individuals a cognitive test and a set of self-responded questionnaires. A study has been conducted to investigate the effects of survey administration mode on survey responses, using data collected in National Assessment of Educational Progress, a large scale education survey in the U.S. In the study, national representative sample of 4th and 8th grade students in the U.S. were randomly assigned to each of the two conditions (paper vs. computer mode). We discuss the methods and analyses used to evaluate whether the cognitive test administered in the digital mode produces results for student populations that are comparable to those obtained on paper mode. In addition, we describe the approaches took to enable comparing results from the two modes, through random groups design and distribution matching.