Evaluating Decision Tree Splitting Criteria for Differential Treatment Effect

• Activity Number: 60
• Type: Topic Contributed
• Date/Time: Sunday, August 4, 2013 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Padraic Neville*+
• Companies: SAS Institute
• Keywords: Decision tree ; Differential response ; Treatment effect

Abstract

Ultimately we seek a method to characterize people who will respond to a treatment. A similar problem is to predict who will respond to a marketing promotion. Many published methods use decision trees, and several papers propose splitting criteria. We recommend evaluating the criteria independently of other choices in the methodology. We apply six proposed criteria for binary treatment and response to simulated data in which the odds ratio changes when an input value exceeds a threshold. The ratio of splits near the threshold to splits near the extremes varies from 0.02 in the worst criterion to 3.74 in the best. Requiring a minimum branch size mitigates the bias towards the extremes, but some criteria remain much more focused on the threshold than others. The best among the six is the maximum of the standardized treatment effect in the two branches.