Boosting in General: Consistency and Minimaxity

• Activity Number: 431
• Type: Invited
• Date/Time: Thursday, August 7, 2003 : 8:30 AM to 10:20 AM
• Sponsor: IMS
• Author(s): Yaacov Ritov*+ and Peter J. Bickel
• Companies: The Hebrew University of Jerusalem and University of California, Berkeley
• Keywords: ADAboost ; classification

Abstract

Boosting, an iterative method for constructing complex classifiers from simple ones has recently been investigated from a statistical point of view, Breiman (2000), Jiang (2002), Lugosi and Vayatis (2002), Zhang (2002), Buhlmann (2002). The results fall into two classes. The first consists of showing that early stopping with relatively unrestricted class of base classifiers, leads to Bayes consistency, shown for L2 boosting by Buhlmann (2002) who compares the result to matching pursuit (Mallat and Zhang, 1993) and classical ADABoost by Jiang (2002) under very stringent conditions. The second, Lugosi and Vayatis (2002), Zhang (2002), consists of establishing rate of convergence results but for penalized boosting. We give a few generalizations of the first class of results to general loss function such as logit boost (Friedman, Hastie, Tibshirani 2001) on the one hand, and on the establish rate results for boosting with thresholding--forming a hierarchy of base classifiers and proceeding from level to level of the hierarchy using thresholding stopping finally using cross-validation--on the other.