Assessing effects of survey mode on healthcare survey responses through experimental manipulation of order
*Alan M. Zaslavsky, Harvard Medical School, Dept. of Health Care Policy 

Keywords: mode effect, healthcare survey, principal stratification

The Consumer Assessments of Healthcare Providers and Systems (CAHPS) surveys of Medicare beneficiaries are conducted initially by mailout-mailback of a paper instrument, with telephone followup of nonrespondents. Differences between mean responses by mail and telephone can occur due to a combination of mode selection (those responding by telephone differ from those responding by mail) and mode response effects (responses by telephone differ from those that would be obtained from the same respondents by mail). We identified these effects using an embedded experiment in which a random subsample of beneficiaries was approached first by telephone and afterwards by mail. Our analysis was conducted in a principal stratification framework, decomposing the universe of potential respondents into mail-only respondents, phone-only respondents, and those who would respond by either mode.