Promenade Upper
Tell Me More: Data Quality, Burden, and Designing for Information-Rich Responses in Web Surveys (303642)
*Lilian Yahng, Indiana UniversityKeywords: web survey design, survey data quality, response burden, respondent motivation, qualitative data
In an environment of limited budgets and the prevalence, ease, and low cost of web surveys, researchers may be increasingly turning to such self-administered options to collect complex, information-rich, and/or qualitative data that traditionally (or ideally) may have called for an interviewer. Thus, following research that suggests clarifying and motivating language improves response quality on open-ended items in web surveys, we devised a randomly-assigned split-ballot experiment that tested the effect of motivating as well as explicitly grateful language in a web survey gauging educational and programmatic activities of health sciences faculty in a large multi-campus university. The questionnaire itself was prima facie burdensome, soliciting not only detailed recall but also written descriptions of activity objectives and outcomes---and this battery was repeated for each activity (up to three) reported. Those assigned to the experimental treatment received additional text on select questions and section transitions that thanked respondents for responses thus far, encouraged further reporting, and implicitly acknowledged the cognitive burden, approximating in this self-administered mode techniques interviewers employ to elicit cooperation and response. Those receiving normal treatment were administered the base questionnaire only, without the additional language. In our analysis we assess the survey data for the treatment groups on item nonresponse, character/word counts, coding of unique thematic elements, and presence of progressive effects both overall and in respondents reporting for more than one activity. We also examine survey duration, breakoffs, and respondents’ own metasurvey comments as proxy measures of perceived burden. Our results may have broader applicability for capitalizing on cognitive and technical aspects of web survey environments and also be of interest to practitioners adapting questionnaires for different modes.