Online Program

Usability testing and evaluation of questionnaire re-design work

Keywords: usability testing, questionnaire design, response burden

Statistics Finland had a two-year project on the evaluation and testing of business web questionnaires, and to redevelop them and give guidelines for future work on questionnaire design and testing. The usability testing was done through cognitive interviews with the concurrent think-aloud method and recording of all interviews. The screen recordings were done with the Dream Screen software, which produced the video clips with audio during the answering process. The video material and usability guidelines guided the re-design work of the web questionnaires afterwards. The results of response burden measurement will be evaluated after re-design work and the focus is in the areas where changes have been made. By using a quantitative data parallel with qualitative it is possible to have more holistic view of the effectiveness of development procedures.

This paper discusses the experiences gained from the testing and from re-designing work, which appeared to be the most challenging part of the work. Moreover the first quantitative results of next data collection round in re-designed questionnaires and the results of response burden measurement give a possibility to assess the effects in versatile manner. An Internet data collection method has been available to all over 65 major and permanent data collections from the end of 2006. This means that the web questionnaire designing was done mostly in the early years of the 2000s. Since then the web questionnaires in general have been developed in many different ways and people’s expectations on the web applications have risen. Although the data collections via the Internet is regarded as successful in many ways, continuous evaluation and development of the web questionnaires are still needed to ensure high quality of the collected data and to keep the respondents motivated to give answers.