Got a Phone Number? Examining the Reliability and Accuracy of Phone Number Append Propensity Models for ABS Samples

• Activity Number: 35
• Type: Contributed
• Date/Time: Sunday, August 9, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Survey Research Methods Section
• Author(s): Trent Buskirk* and Kristen Olson
• Companies: Marketing Systems Group and University of Nebraska - Lincoln
• Keywords: ABS Sample ; Appended Data ; Propensity Modeling ; Machine Learning ; Model Accuracy ; Coverage Bias

Abstract

Address-based samples allow auxiliary data to be linked to addresses via geographic coordinates. These variables are used for nonresponse adjustments and follow-ups. One common design, an ABS Sample with Phone Follow-Up, sends mailings to all addresses, and then makes phone calls to addresses with phone numbers appended and sends mailings to those without. Differences in completion rates and respondent characteristics have been reported for addresses with and without phone appends, but it is not clear whether this is related to initial differences in having an appended phone number. In this paper we identify variables related to phone append status. Cross-validated accuracy, error rates and variable importance measures are presented from various machine learning models predicting whether an address has a phone append (about 45% of addresses) from over 500 candidate variables appended to a sample of 1 million records randomly chosen from an ABS sampling frame. The results will allow researchers to understand how initial phone append status is related to various socio-demographic and economic variables, and could potentially affect coverage and nonresponse biases.