Ensemble Classification Based on Generalized Additive Models

• Activity Number: 356
• Type: Contributed
• Date/Time: Tuesday, August 3, 2010 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Koen W. De Bock and Kristof Coussement*+ and Dirk Van den Poel
• Companies: Ghent University and Université Catholique de Lille and Ghent University
• Keywords: Data Mining ; Classification ; Ensemble Learning ; GAM ; UCI

Abstract

Generalized additive models (GAMs) are a generalization of generalized linear models (GLMs) and constitute a powerful technique which has successfully proven its ability to capture nonlinear relationships between explanatory variables and a response variable in many domains. In this paper, GAMs are proposed as base classifiers for ensemble learning. Three alternative ensemble strategies for binary classification using GAMs as base classifiers are proposed: (i) GAMbag based on Bagging, (ii) GAMrsm based on the Random Subspace Method (RSM), and (iii) GAMens as a combination of both. In an experimental validation performed on 12 data sets from the UCI repository, the proposed algorithms are benchmarked to a single GAM and to decision tree based ensemble classifiers (i.e. RSM, Bagging, Random Forest, and the recently proposed Rotation Forest).