|
Abstract:
|
Class imbalance commonly occurs in many real classification problems. Traditional evaluation metrics such as misclassification error rate are not very informative when the data is highly imbalanced. As an alternative, recall and precision are often used to assess the performance. In this work, we formally introduce the notion of optimal classifiers in terms of precision and recall over an arbitrary model space. It is argued that any classification method implemented over the given model space should target the optimal classifiers, leading to a minimal requirement which we call $\mathcal H$-consistency. Next, we propose a novel two-step procedure to estimate optimal classifiers and show its $\mathcal H$-consistency over a general model space, which allows for possible model mis-specification. Efficient computational methods are also developed for the proposed two-step procedure. Moreover, we show that cost-sensitive learning, a popular method for imbalanced data classification, is not $\mathcal H$-consistent, and may miss some optimal classifiers when model is incorrectly specified. We demonstrate the efficacy of our proposed method through both simulations and a real data analysis.
|
