How Should We Adapt Complex Social Research Questionnaires for Mobile Devices? Evidence from UK Surveys and Experiments (303303)

Keywords: mobile devices, smartphones, mobile optimization, device agnostic design, mobile questionnaire design, grids, usability testing

A growing proportion of people are choosing to complete online social surveys using mobile devices, including smartphones. As ownership and use of mobile devices continues to grow, it is crucial that we adapt survey design to enable people to complete questionnaires on their device of choice, without impacting negatively on the respondent experience.

This paper draws on data from a range of UK social surveys to show how device use impacts on survey experience and behavior. Results from usability testing on Understanding Society (the UK Household Longitudinal Study)---where respondents completed the survey on smartphones---allows us to vividly illustrate the questionnaire design challenges associated with completing social surveys on smartphones. Based on this testing, and other evidence from social and commercial research, we highlight core principles to improve the experience of completing surveys on mobile devices.

We also present results from two new experiments, to help inform questionnaire design decisions. The first experiment focuses on the use of grids, which past studies show can be problematic for those completing on smartphones (e.g., McClain and Crawford, 2013), and compares traditional grids, dynamic grids and item-by-item formats. The second experiment assesses the impact of question length on response, comparing detailed questions with simplified and shortened versions. This provides important evidence for researchers as they adapt interviewer administered surveys to online (including mobiles), and typically need to reduce question length to reflect this new format.

With social studies increasingly moving online, we need to better understand the ways that people want to complete surveys on mobiles, how different devices can impact on behavior and the questionnaire design challenges to overcome. This paper adds important evidence to these debates as we seek to effectively design and adapt social surveys to support completion by mobile devices.