A Quadratic Programming Algorithm for Reducing the Size and Improving the Performance of an Ensemble Model

• Activity Number: 236
• Type: Contributed
• Date/Time: Monday, August 1, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Computing
• Author(s): Jie Xu*+ and J. Brian Gray
• Companies: University of Alabama and University of Alabama
• Keywords: bagging ; boosting ; machine learning ; predictive modeling ; random forests

Abstract

Ensemble models, such as bagging, random forests, and boosting, have better predictive accuracy than single classifiers. These ensembles typically consist of hundreds of single classifiers, which makes future predictions and model interpretation much more difficult than for single classifiers. According to Breiman (2001), the performance of an ensemble model depends on the strengths of the individual classifiers in the ensemble and the correlations among them. In this article, we propose a new method based on quadratic programming that uses information on the strengths of, and the correlations among, the individual classifiers in the ensemble, to improve or maintain the predictive accuracy of an ensemble while significantly reducing its size.