Thursday, November 10
Pretesting Methods
Thu, Nov 10, 3:30 PM - 4:55 PM
Orchid AB
Designing and Testing Question Types

Measuring Political Identity and Religiosity: Does Mode or Presentation Matter? (303146)

*Dag F Gravem, Statistics Norway 

Keywords: mode effects, response scales, cross-cultural surveys

Rating scales in their many shapes and forms is a topic that has been examined from different perspectives throughout the history of survey methodology. In a recent article, Yan and Keusch point out that, despite this, the direction, presentation, or order of a rating scale is a design feature that remains understudied. As a contribution in this respect, they present an experimental study from an interview survey where an 11-point end-labeled unipolar rating scale was being used. They found that the direction of the scale significantly affected the respondents’ rating of the object that received high ratings. They argue that such direction effects are due to respondents’ use of anchoring-and-adjustment heuristics rather than satisficing.

In our study, we wish to investigate the possible effects of visual presentation on two 11-point scales in a web study, one unipolar and the other bipolar. The Norwegian Election Surveys have for many years been an interview survey conducted biannually shortly after municipal or national elections are completed. In this survey, respondents have been asked to place themselves politically on a bipolar scale, where 0 means on the far left politically, and 10 means on the far right politically. The respondents are also asked about how important religion is in their life, on a unipolar scale where 0 means not important at all, and 10 means that religion is of very high importance to them.

As of 2015, the Election Survey will be conducted in web mode instead of as an interview survey in mixed CATI/CAPI mode. The shift from interviewer-administered modes to self-administered modes, and to web in particular, has been a general trend among NSIs for the last several years, as noted by Blanke et al. The main reason for this is that money must be saved in times of shrinking budgets. When such mode shifts occur, researchers should be aware of questions design features that can result in mode effects.