Online Program

Evaluating Non-response Bias in Organizational Surveys Using Response Timing:

Keywords: employee surveys, nonresponse bias, response timing

Survey non-response and item non-response have long been concerns among survey researchers. These problems occur as frequently in organizational surveys where a census is attempted as in sample surveys of the general public, and have the same potential to introduce bias into the survey results. Organizational researchers must also frequently respond to such concerns from clients, which are sometimes articulated in terms of “how far off are we from our ‘true’ score given our less-than-ideal response rate to our employee survey?” A practical strategy for answering such questions is to use late responders to a survey as proxy for non-responders in assessing the extent of non-response bias. Drawing on Gallup’s workplace research from more than 30 years involving hundreds of organizations, thousands of workgroups and millions of employees, this paper asks two related questions. First, how does response timing affect measures of employee engagement? Second, how does response timing affect data quality measured as item-nonresponse? To answer these questions, the paper tests a series of multilevel models (generalized linear mixed models).