Mode Effect Analysis and Adjustment in a Split-Sample Mixed-Mode Web/CATI Survey

• Activity Number: 606
• Type: Contributed
• Date/Time: Wednesday, August 7, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Stanislav Kolenikov*+ and Courtney Kennedy-Shea
• Companies: Abt SRBI and Abt SRBI
• Keywords: mode effect ; mixed mode survey ; multiple imputation ; non-response ; social desirability bias ; split-sample design

Abstract

We analyze the results of a national survey collected in two modes: SAQ on the web, followed by personal CATI of web non-respondents. We apply regression and implied utility-multiple imputation mode effect adjustments. Since some items may exhibit mode effects, such as social desirability bias, a randomized split-sample design has been built into the study to allow for a rigorous comparison of the item response distributions in the two modes. A logistic model for Yes/No responses or an ordinal logistic model for Likert scales was fit to the data with explanatory variables that included demographic variables and the mode indicator. The regression mode effect adjustments zeroed out the mode variable and formed predictions using the estimated regressions coefficients. Another mode adjustment is based on econometric framework of implied utilities in logistic regression modeling, in which the alternative is chosen with the greatest utility. The latter was treated as a version of multiple imputation. We found the community involvement items and experiencing a major financial crisis recently to exhibit the strongest mode effects.