Friday, November 11
Questionnaire Design
Fri, Nov 11, 10:30 AM - 11:55 AM
Hibiscus B
Mobile and Web Survey Design

Question Grouping and Matrices in Web Surveys (303628)

*Ipek Bilgen, NORC at the University of Chicago 
Erin Fordyce, NORC at the University of Chicago 
Michael J. Stern, NORC at the University of Chicago 
David Sterrett, NORC at the University of Chicago 

Keywords: Web Surveys, Questionnaire Design, Screen Size, Smartphones, Response Rates, Data Quality, Auxiliary Data

Due to the time and cost efficiencies associated with web surveys as well as the increase in internet use among American households, survey organizations in the U.S. are increasingly using web surveys. Additionally, the increase in smartphone and tablet use makes the web surveys an attractive mode to be used in addition to the more conventional data collection modes. Hence, web questionnaire design issues, as well as device-specific visual design issues have become an important yet poorly understood aspect of survey research. One of the questionnaire design issues relevant to web surveys is question grouping and utilization of grid matrices. Studies on web surveys have illustrated that the question grouping decisions may be associated with break-offs, item nonresponse, and questionnaire completion times. In this study, we are comparing item nonresponse, questionnaire completion times, and break-offs among three version of the question groupings and in different devices. In order to do so, we have implemented an experiment among 4,000 sampled addresses including a series of survey invitation mailings including a link to the web survey, as well as the instructions on how to complete this survey. In this experiment, three different visual groupings of several health related questions are examined: 1) item-by-item version in which each question is asked on a different screen; 2) small matrix version in which questions are presented via grids on two screens; and 3) full matrix version in which questions are presented via one grid on one screen. Our research aims to shed light on the impact of visual complexity of matrices and utilization of question grouping and grids in web surveys, as well as the comparability of different visual groupings among multiple devices with different screen sizes.