Comparing Random Forests and Random Multinomial Logit to Rotation Forest and the New Rotation Multinomial Logit

• Activity Number: 507
• Type: Topic Contributed
• Date/Time: Wednesday, August 5, 2009 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistics and Marketing
• Author(s): Anita Prinzie*+ and Dirk Van den Poel
• Companies: Manchester Business School/Gent University and Ghent University
• Keywords: classifier ensembles ; random forests ; MultiNomial Logit ; bagging ; analytical Customer Relationship Management ; variable selection and feature reduction

Abstract

Classifier ensembles are popular within machine learning and have recent applications in aCRM. This research focuses on bagged classifiers. Random Forests (Breiman, 2001) builds a 'forest' of decision trees splitting at each node on the best feature out of a random subset of the feature space. Prinzie and Van den Poel (2008) generalized the Random Forests principles and introduced Random MultiNomialLogit, combining a forest of MNLs estimated with randomly selected features. Rotation Forest (Rodríguez and Kuncheva, 2006) replaces random feature selection as a source of ensemble diversity with feature extraction. The feature set is randomly split into K subsets and the K axis PCA rotations form the new features of the decision tree base classifier. We present Rotation MNL to assess the value of the rotation approach for an ensemble with MNL as base classifier.