Online Program

Adjusting Survey Mode Differences: Illustration of a Linear Equating Method (303022)

Keywords: mode differences, adjustment, linear equating, equality of scores

Consistent differences have been found in survey responses from web verses phone mode. The present study was designed with two major goals: a) to determine the extent of response differences between phone and web and b) to derive an adjustment formula to adjust the phone scores to match with web scores. In this experiment, well-being scores from a random sample of 20,000 phone Gallup Poll survey were compared with 16,000 web Gallup random panel survey responses. The phone scores were significantly higher than the web scores. We applied a linear equating method in which scores that were at equal distance from their means in standard deviation were set equal and an adjustment formula was derived. Further, we explored the differential effect by mode across various subpopulations using a generalized linear modeling method and fine-tuned the adjustment equations. The final formula was cross-validated in two other large samples. The application of this linear equating method, which comes from very basic statistical concept of Z-score, is not restricted to just mode effects; it can be applied to any two surveys built to the same content and statistical specifications to adjust for differences.