Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 34 - Current and Future Challenges in Diagnostic Device Evaluation
Type: Topic Contributed
Date/Time: Monday, August 3, 2020 : 10:00 AM to 11:50 AM
Sponsor: Section on Medical Devices and Diagnostics
Abstract #313433
Title: Proper and Improper Summary Measures of Classification Performance
Author(s): Andriy Bandos*
Companies: University of Pittsburgh
Keywords: Classification performance; ROC; proper summary measure; misclassification rate; Kappa coefficient; Youden’s index

Summary measures of classification performance are used widely in the tasks related to classification, prediction, decision making, and classifier optimization. They include such generally used summary statistics as odds ratio and kappa coefficient, summary indices well-known in the decision making field such as Youden’s index, as well as more technical indices such as proximity to the perfect classifier, misclassification rate, etc. In addition to the classic summary measures, new performance indices are continued to be regularly proposed to emphasize particular characteristics of a targeted classification task. We demonstrate that some of the popular summary indices have intrinsic ability to lead to objective errors in ranking classifiers. We develop a formal concept of “improper” functional summary of classification performance, and provide a practical criterion for evaluating summary indices. The concept and tools are then used to determine the proper/improper nature common summary indices. The developed framework enables identifying summaries indices that should be used with caution, and provides guidelines for development of robust approaches for ranking classifiers.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program