Best Practices of Usability Testing Online Questionnaires at the Census Bureau: How Rigorous and Repeatable Testing Can Improve Online Questionnaire Design (303154)

Keywords: usability testing methods and tools, eye tracking, remote testing, think-aloud

As more questionnaires move online, new challenges have emerged with evaluating these questionnaires. These challenges include testing the effectiveness of both the survey question and response options along with the form factors that are associated with the automated questionnaire.

Usability testing evaluates how a user interacts with an interface focusing on efficiency, effectiveness, and user satisfaction while accomplishing tasks (ISO standard 9421). In usability testing of online surveys, the task the respondent has to accomplish is to complete the survey while navigating a graphical user interface. There are three primary goals of the usability evaluation. First, the respondent has to accomplish the task of completing the survey within a reasonable amount of time. Second, like cognitive testing, usability testing must assess whether the respondent understood the question, found and selected the correct response option, and answered all the required questions. Third, the respondent has to answer the survey without breaking-off. Using data from the usability testing conducted on various economic, business, and demographic surveys, we will share how different usability testing study designs have been used to improve the online surveys at the U.S. Census Bureau. While there has always been a respondent completing an online form while being observed by an administrator, protocols for each test vary according to the research goals. Examples of varying methodology include the use of cognitive probing techniques, such as think-aloud, concurrent or retrospective debriefing probes, vignettes, voice and screen recording, eye-tracking, in-lab and remote testing, as well as the use of low-to-high fidelity prototypes and iterative testing. We will also touch on the use of different communication techniques to relay the usability issues to the survey owner. We will highlight the most effective techniques we have found to accomplish the three goals of usability evaluations.