Interaction Detection with TreeNet Boosted Tree Ensembles for Predictive Modeling

• Activity Number: 295
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Dan Steinberg*+
• Companies: Salford Systems
• Keywords: gradient boosting ; interaction detection ; Multiple Additive Regression Trees ; boosted trees ; model simplification ; constrained trees

Abstract

Recent advances in machine learning technology make it possible to determine definitively whether or not interactions of any degree need to be included in a predictive model. We can thus establish conclusively, for example, for a given set of predictors, that an additive model (one with no interactions) cannot be improved upon with interactions. Or alternatively, one might prove that a predictive model with interactions will outperform a model without them. Further, we can now identify precisely which interactions are supported by the data, and also the degree of interaction, even in very high dimensional data. The tools we use to acheive these results are extensions of Stanford University Professor Jerome Friedman's TreeNet, developed by the authors and embedded in SPM®, the Salford Systems Predictive Modeler software suite. We illustrate the concepts in the context of a real world regression model where we are quickly able to identify all the important interactions with a modest number of boosted tree ensemble models.