Using Genetic Algorithms to Improve the Construction of Classification, Regression, and Survival Tree Models

• Activity Number: 253
• Type: Invited
• Date/Time: Tuesday, August 10, 2004 : 2:00 PM to 3:50 PM
• Sponsor: Business and Economics Statistics Section
• Author(s): J. Brian Gray*+ and Guangzhe Fan
• Companies: University of Alabama and University of Alabama
• Keywords: bagging ; CART ; data-mining ; genetic algorithm ; random forests

Abstract

Tree-structured modeling is a valuable tool for predictive modeling and data mining. Traditional tree-growing methodologies such as CART suffer from "greediness," i.e., local optimizations at nodes to be split in a decision tree do not always result in global optimization of the tree model. CART solutions are also sensitive to perturbations in the data and can vary greatly across different training sets sampled from the same data. Bagging, random forests, and other ensemble techniques have improved on the predictive performance of CART, but they do not have the interpretability of single-tree models. We describe the TARGET (Tree Analysis with Randomly Generated and Evolved Trees) method of constructing tree models for classification, regression, and survival data. Empirical evidence suggests that TARGET produces smaller trees with better predictive performance than CART. TARGET solutions are also found to be more stable than CART solutions across different training samples from the same data.