Nonresponse Bias Adjustment in Establishment Surveys: A Comparison of Weighting Methods Using the Agricultural Resource Management Survey (ARMS)

• Activity Number: 116
• Type: Topic Contributed
• Date/Time: Monday, July 30, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Section on Government Statistics
• Author(s): Morgan Earp*+ and Jaki McCarthy and Frauke Kreuter and Roger Tourangeau
• Companies: Bureau of Labor Statistics and National Agricultural Statistics Service and Joint Program in Survey Methodology and Westat
• Keywords: Nonresponse Bias ; Classification Trees ; Logistic Regression ; Propensity Scores ; Calibration ; Nonresponse Adjustment

Abstract

There are numerous ways to address nonresponse bias adjustment in surveys; two such methods are calibration weighting and propensity score models. Calibration is a viable technique when good external benchmarks exist; however, good external benchmarks are not always available. An alternative method to calibration is to use propensity scores to adjust for nonresponse. There are at least three main modeling techniques used to create propensity scores, but little if any research has focused on which methods provide the best propensity scores in terms of nonresponse adjustment. This paper compares calibration weights with three propensity score adjustment methods. One propensity weight is based on logistic regression models; the other two are based on classification trees (using either a single or an ensemble tree approach).