On the Characterization of a Class of Fisher-Consistent Loss Functions and Its Application to Boosting for Hierarchical Outcomes

• Activity Number: 367
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Matey Neykov*+ and Tianxi Cai
• Companies: Harvard University and Harvard University
• Keywords: Fisher-Consistency ; Boosting ; Classification ; Hierarchical Outcomes

Abstract

Fisher-Consistent Loss functions play an important role in Decision Making Theory. In this talk we focus on a characterization of a broad class of Fisher-Consistent Loss functions. Following closely Zou et al (New Multicategory Boosting Algorithms Based on Multicategory Fisher-Consistent Losses), we generalize existing results on the class of loss functions that achieve Fisher consistency. We also propose a generic iterative procedure that converges to the minimizer of the loss function and illustrate the new proposal with a generalized boosting algorithm. We further extend our proposal to enable efficient classification of hierarchically structured multiple outcomes with a hierarchical boosting algorithm. We demonstrate that the new proposal, leveraging the tree structure of the outcomes, attains optimal classification accuracy with respect to a class of pre-specified cost parameters.