A Comparison of Fully Labeled and Top-Labeled Grid Question Formats (303159)

Keywords: grids, mail survey, eye tracking, experiment

Grids are common in mail and web surveys because they reduce the redundancy of listing the response options and question stem content for every item and save space. However, the grid format is challenging to process and produces high rates of item nonresponse, straightlining, and breakoffs (Couper et al., 2013; Tourangeau et al., 2004). Surveyors can use hover, shading, or greying within grids to help respondents in web surveys (Kaczmirek, 2011; Tourangeau et al., 2013), but not in mail surveys. One possible way help in mail is to repeat the response option labels in each row with the answer spaces (Dillman 1978). This format eliminates the need for vertical processing; however, it yields a visually dense display that may increase skips of the entire grid. In an initial, limited experimental test of this format, it reduced item nonresponse rates compared to a traditional grid but did not impact rates of skipping the grid entirely (Smyth et al., 2014). In this paper we report the results of more extensive experimental comparisons of fully versus top labeled grid formats in two surveys. The first is the National Health, Wellbeing, and Perspectives Survey, a mail survey conducted during summer 2015 (n=6,000 sampled; n=1002 responded; AAPOR RR1=17.8%) that contained seven different grid items with three types of response options (numeric frequencies, vague quantifier frequencies, and agree/disagree). We will compare rates of skipping the grid entirely, item nonresponse rates within the grid, straightlining, response distributions, and scale reliability across the two formats. The second set of data is from laboratory eye tracking experiments conducted in 2013. 167 university students and general population members completed a web survey with eye tracking. They received either fully labeled or top-labeled grids for 2 items. We will analyze the eye tracking data for differences in visual processing (e.g., whether the fully labeled grid reduces vertical eye movements).