|
Abstract:
|
As a means of quality control, survey paradata have been used to detect problematic interviews and responses. In particular, response time (RT) to a specific item, a particular question module, or the entire instrument are considered as informative measures for identifying performance outliers, such as speeders, draggers, or potentially falsified interviews. However, the quality assurance techniques used in the survey industry seems to be established based on practical experiences. There is scant research on determining cutoff thresholds for RT measures and the implications for detecting response outliers. In this study, we evaluated various behind-the-scenes techniques for using RT to detect response outliers. We found that using Tukey (1977) method to analyze normalized RT distribution to determine proper IQR cut-off threshold for response outlier detection may be the more robust technique for identifying RT outliers. We have replicated the technique in multiple studies and produced consistent execution for quality assurance. We further suggest how to examine additional satisicing patterns for quality assurance when RT outlier identification may not be effective.
|
