Combining Usability and Cognitive Tests to Optimize the 2021 Census for Mobile Devices (303368)

Keywords: census, usability, cognitive, testing, questionnaire, development, household, England, Wales, mobile device, smartphone, tablet

The Office for National Statistics is undergoing transformation to bring surveys and the Census into the digital era. This modernization includes developing surveys that can be completed online. The next England and Wales Census will be in 2021 and responses collected predominantly online. Question development will consider various modes of data collection from a desktop computer to a smartphone or tablet. This aims to increase response rates by enabling users to respond on any device, therefore improving usability. To establish the feasibility of using mobile devices as a means of data collection, the 2011 online census questions were tested for usability with members of the public as a baseline for redesign. User research techniques, including observation and questioning participants on elements of their experience were used to observe participants completing the 2011 questionnaire on their own device and in their own homes. Pre-defined probes were used to gather feedback on general usability and question design and observational probes were used to pick up on spontaneously occurring problems.

Findings indicated that the 2011 questionnaire could be completed on a mobile device but revealed problems with usability and participant burden. Entering a 20 digit security code was particularly problematic on a mobile device due to the need to switch the device keyboard between numbers and upper and lower case letters. The number of response options optimized for paper was not acceptable on a mobile device as the amount of scrolling interfered with navigation. Recommendations included stream-lining the introduction to enable respondents in larger households to select a suitable device, thus reducing break-off rates. Problematic questions for redesign were identified, as were security and privacy concerns about submitting information on a mobile device. The findings fed into further development and further iterations were cognitively tested and tested for usability.