Strategies for Extracting Knowledge from Ensemble Classifiers Based on Generalized Additive Models

• Activity Number: 236
• Type: Contributed
• Date/Time: Monday, August 1, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Koen W. De Bock*+
• Companies: IESEG School of Management
• Keywords: ensemble classification ; generalized additive models ; GAMens ; model interpretability ; bagging

Abstract

In recent literature, ensemble learning has demonstrated superior performance in a multitude of applications. However, their increased complexity often prevents qualitative model interpretation. In this study, GAMensPlus, an ensemble classifier based upon generalized additive models (GAMs), in which both performance and interpretability are reconciled, is presented and evaluated. The recently proposed GAMens, based upon Bagging, the Random Subspace Method and semi-parametric GAMs as constituent classifiers, is extended to include two instruments for model interpretability: generalized feature importance scores, and bootstrap confidence bands for smoothing splines. These elements allow insight into (i) the relative importance of features within the model, (ii) the nature of the relationship between each feature and the outcome, and (iii) the reliability of these estimated trends at different regions within the range of the feature. In an experimental comparison on UCI and simulated datasets, the strong classification performance of the proposed algorithm versus a set of well-known benchmark algorithms as well as properties of the strategies for interpretability are demonstrated.