34 – Mode Effects
Mode Effects in American Trends Panel: A Closer Look at the Person-Level and Item-Level Characteristics
Stanislav Kolenikov
Abt SRBI
Kyley McGeeney
Pew Research Center
Scott Keeter
Pew Research Center
Nancy Mathiowetz
Andrew Mercer
Westat
Courtney Kennedy
Abt SRBI
American Trends Panel is a probability panel with RDD recruitment developed by by Pew Research Center and Abt SRBI. Panel surveys have been conducted on different modes in different waves, including web for most panel members, and mail or phone for those who do not have access to the Internet. We analyze the results of the July 2014 wave (Wave 5) that included a comprehensive, large-scale mode-of-interview experiment that randomly assigned respondents to telephone and web modes, with approximately 1,500 respondents in each mode. To quantify the contributions to the mode effects of the different question characteristics in the 75-question instrument, we build a cross-classified mixed model with effects of person and question characteristics to identify the properties of survey questions that make them susceptible to mode effects, as well as the demographic groups that tend to exhibit mode effects. We discuss how the decomposition of the total survey error and explained variance helps identifying the properties of the questions that are associated with the mode effects, such as question format, topic, and the potential impact of social desirability.