Friday, November 11
Data Quality and Measurement Error
Fri, Nov 11, 4:00 PM - 5:25 PM
Hibiscus B
Understanding the Role of Questionnaire Design in Measurement Error

How Question Format and Mode Can Affect the Number of Bicycles in a Household: An Evaluation and Pretest Study (303371)

Vivian Meertens, Statistics Netherlands 
*Rachel Vis-Visschers, Statistics Netherlands 

Keywords: question format, mode effects, pre-test

Since 2010, the mobility survey has had a mixed-mode design (Web-CAPI-CATI), with stable response patterns and distributions between modes. Until 2012, the survey produced stable time series of how many bicycles there are in a household and how many persons own a bicycle. In 2013, several questions about electronic bicycles (e-bikes) were introduced. The introduction of these new questions caused a mode effect. For instance, in Web it appeared that 20% less people owned a bike, while in CAPI and CATI, the trend was unchanged.

To investigate what caused the mode effect, qualitative research was conducted, including literature review, expert review, cognitive interview, and qualitative test interviews.

Results showed that order and context effects, misinterpretation, and interviewer effects affect the number of bicycles in a household and the ownership of bicycles.

Several alternative questions and respond options were developed and pre-tested. The questions that came out of the pre-test as the most promising were included in the actual survey in 2015.

During the presentation, the execution and results of the research project and pre-test are discussed. And, finally, the results of the fieldwork of the new questions are presented. What happened to the time series …?