Interactive Design Features in Web Surveys: Instant Feedback to Respondents in Grid, Check-All, and Open-Ended Questions (303583)

Keywords: satisficing, measurement error, question answer process

Web surveys offer the opportunity to make use of interactive features such as nonresponse or speeding prompts in order to remind respondents of problematic response behaviors. Typically, such prompts are triggered after respondents have clicked on the “Continue” button, which is after they have completed the current question and have already intended to continue with the next question. Previous research concerning speeding and nondifferentiation prompts in grid questions has demonstrated that such delayed feedback provided after clicking on the “Continue” button is less effective than instant feedback provided while respondents are still in the process of answering the current question. It has been speculated that feedback is most effective if it is provided in temporal coincidence with the respondent behavior addressed in the feedback. In this paper, we will extend the instant feedback approach to other question types often used in questionnaires. In addition to grid questions, we will focus on check-all-that-apply and open-ended questions. We will assess whether and to what extent instant feedback is capable to influence respondent behavior und ultimately improve data quality. Because of the increasing use smartphones in Web surveys we compare respondents using mobile devices and desktops/notebooks. While we consider nondifferentiation in grid questions, we assess respondent compliance with feedback in check-all-that-apply questions and the wealth of information obtained in open-ended narrative questions. Results are based on field-experimental studies implemented in large-scale Web surveys conducted from 2014 to 2016 among study applicants of a German university. In addition to the practical implications of reducing undesired response behaviors, we are particularly interested in the optimal timing of instant feedback and more generally in the mechanisms by which its use influences the question-answer process.