How the Device Screen Size Affects Data Collected in Web Surveys (303136)

Keywords: web surveys, mobile devices, screen size, questionnaire optimization, survey methodology, data quality, data comparability

Survey participation by means of mobile devices is quickly spreading in many countries. This new kind of participation stimulated a lot of methodological interest. Indeed, mobile devices (and smartphones, mostly) have different characteristics in comparison to PCs, in particular a higher portability and smaller screens.

Previous research has been focused mainly on experiments comparing PC and mobile survey participation. Only few studies specifically took into consideration the screen size of the mobile devices as a factor that can affect the quality of web surveys data. However, the size of the screens was always considered a key aspect in the framework of web surveys. This paper aims at evaluating how the screen size of the mobile devices can affect the results and the data quality in web surveys.

Our analysis, first, takes into account two wide categories: PCs (i.e., “big screen”) and smartphones (i.e., “small screens”). Then, within the data collected on smartphones, we evaluate the effect of the device size on several indicators such as answers consistency across waves, failures to the Instructional Manipulation Check (IMC) question, and completion times.

We use data from an experiment implemented in 2015 by means of a two-wave survey proposed to more than 1,000 panelists of the online access panel Netquest, in Spain. In the two waves we proposed the same questionnaire, but respondents were assigned randomly to different devices (PC or smartphone).

We expect the size of the screen to affect the quality and the comparability of the collected data. When the screens are really small, the answer inconsistencies, failure of the IMC and completion times are expected to be higher, mainly when the survey is not optimized for mobile participation.

Overall, our findings will provide a more precise idea about the comparability and the relative quality of data collected by means of web surveys depending on the size of the device.