Bridging Cost-Sensitive and Neyman-Pearson Paradigms for Asymmetric Binary Classification

• Activity Number: 301 - Application of Asymmetric Classification and Multiple Testing
• Type: Invited
• Date/Time: Wednesday, August 11, 2021 : 3:30 PM to 5:20 PM
• Sponsor: WNAR
• Author(s): Wei Vivian Li* and Xin Tong and Jingyi Jeessica Li
• Companies: Rutgers, The State University of New Jersey and University of Southern California and UCLA
• Keywords: binary classification; cost-sensitive learning; Neyman-Pearson classification; asymmetric classification; misclassification cost; type I error

Abstract

Asymmetric binary classification problems, in which the type I and type II errors have unequal severity, are ubiquitous in real-world applications. To achieve a desired classification performance, researchers have proposed two paradigms to deal with such asymmetry: cost-sensitive learning and Neyman-Pearson classification. Since the two paradigms have strength and weakness in different aspects, it remains unclear which paradigm to use in each specific asymmetric classification problem. Our work aims to address this challenge by bridging cost-sensitive learning and Neyman-Pearson paradigms from the perspective of controlling the population type I error. In this article, we for the first time discuss the methodological connection between the two classification paradigms. and we have identified two special cases when the two paradigms lead to equivalent classifiers with the same population type I error. We further propose a TUBE algorithm to estimate the type I error upper bound of cost-sensitive classifiers, and demonstrate that the TUBE algorithm improves cost-sensitive learning by helping objectively select the misclassification costs.