In Which Direction Do You Think This Scale Should Go? The Effect of Assumed Scale Direction on Survey Responses (303322)

Keywords: scale direction, survey responses, scale direction effect, measurement error

Survey literature has documented the impact of scale direction on survey responses. However, there exist neither agreed-upon best practices nor industry standards when it comes to scale direction. In addition, it is not clear whether or not survey respondents think there is an inherent order to a response scale and if yes, in what order. It is also not clear how respondents process a scale that runs in the opposite direction to what they assume. To answer these research questions, we conducted an experiment in a health and lifestyle survey. We randomly assigned respondents to scales that run either from high to low or from low to high. Towards the end of the survey, we asked respondents in what order they think the response scales should go. This paper will present the results of this experiment. We will first show the direction assumed by respondents for a frequency scale and an agreement scale. We then will examine the impact of the assumed scale direction on their answers. Lastly, we will examine the interaction between the assumed scale direction and the actual scale direction assigned to respondents on their answers. Implications of the results will be discussed.