AdaBoost from the Perspective of Importance Sampling

• Activity Number: 436
• Type: Contributed
• Date/Time: Wednesday, August 5, 2009 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Qingzhao Yu*+
• Companies: Louisiana State University Health Sciences Center
• Keywords: AdaBoost ; Bagging ; CART ; Ensemble Learning ; Gradient-Descent Boosting

Abstract

We motivate the success of AdaBoost in classification problems by appealing to an importance sampling perspective. Based on this insight, we propose the Weighted Bagging algorithm, a regularization method that naturally extends AdaBoost to solve both classification and regression problems. Weighted Bagging uses a part of the available data to build models, and a separate part to modify the weights of observations. The method is used with CART and is compared with AdaBoost, Boosting, Bagging, Random Forest and Support Vector Machine. We apply these methods to some real data sets and report some results of simulations. These applications and simulations show the effectiveness of Weighted Bagging.