Generating Forests of Tree-based Models by Permuting the Model-building Process

• Activity Number: 34
• Type: Contributed
• Date/Time: Sunday, August 7, 2005 : 2:00 PM to 3:50 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Bret Musser*+
• Companies: Merck Research Laboratories
• Keywords: recursive partitioning ; model building ; gene expression ; tree-based models

Abstract

In any model-building process, relationships between predictor variables cause complications. In "feature selection" approaches such as subset regression and recursive partitioning (RP), correlated predictors may be mutually substitutable, may make additive contributions, or may show synergy. As with stepwise regression methods, these features are rarely discovered in recursive partitioning models as most RP methods generate a single tree as "the" answer. As in subset regression, there is a solution. This is to produce not a single tree, but a "forest" of models that fit the data. Harvesting knowledge from the forest is much harder than generating it, however. Not only may visually different trees be identical in the sense of generating identical rules, but different rules may lead to identical sets of terminal nodes given correlated predictors. The first case reflects purely topological issues; the second the possibility of alternative models. The methods of this talk focus on permuting the model-building process without changing the input data. The methods are applied to understanding the relationship between DNA microarray data and outcome of breast cancer.