Friday, November 11
Fri, Nov 11, 9:45 AM - 10:25 AM
Promenade Upper
Friday Poster Session, Part 1

Impact of Response Scale Direction on Survey Responses in a Web Survey (303115)

Mengyao Hu, University of Michigan 
Florian Keusch, University of Mannheim 
*Ting Yan, Westat 

Keywords: scale direction, scale labeling, scale length, question type, response effect, questionnaire design

Survey literature has demonstrated that many design features of response scales (e.g., number of scale points, numeric and verbal labels) affect how survey respondents process the scale and use these features to construct their responses. One understudied design feature is the direction of a response scale. Existing research shows scale direction could affect survey responses by drawing answers closer to the start of the scale, producing scale direction effects. However, there are two important gaps in the literature on scale direction effects. The first gap is the lack of attention to the moderating effects of scale- and question-level characteristics on scale direction effects. The second gap is the lack of attention to the working mechanism of scale direction effects. To fill the two gaps, we conducted a methodological experiment crossing four factors in a full factorial design---the direction of response scales, length of scales, labeling of scales, and type of scales. We will first analyze the impact of all four experimental manipulations on scale direction effects. We will also investigate the impact of scale direction on validity and reliability by conducting structural equation modeling analyses. Our study will allow us to tease out scale and question features that moderate scale direction effects, as well as provide practical guidance for question writers on the direction of response scales.