Is There a Universe Beyond Sensitivity and Specificity in Evaluation of Diagnostic and Prognostic Performance?

• Activity Number: 450
• Type: Topic Contributed
• Date/Time: Wednesday, August 6, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Section on Medical Devices and Diagnostics
• Author(s): Michael Pencina*+
• Companies: Duke University
• Keywords: diagnostic ; prognostic ; discrimination ; risk

Abstract

Sensitivity and specificity are the most common threshold-based measures of diagnostic and prognostic performance. They also serve as building blocks for more global measures of performance, including the area under the receiver operating characteristic curve (AUC). In recent years many new measures of performance have been proposed in the global as well as threshold-specific setting. We show that the vast majority of the new proposals that gained traction in popular use are related to sensitivity and specificity. Assuming multivariate normality, we relate global measures of model performance (AUC, discrimination slope) and threshold-based measures (sensitivity and specificity, Youden's index, net benefit, relative utility). We observe that unless rules with very good specificity are desired, the change in AUC does an adequate job as a predictor of change in threshold-based measures. However, stronger or more numerous predictors are needed to achieve the same increment in the AUC for baseline models with good versus poor discrimination. Furthermore, markers which increase AUC may decrease sensitivity or specificity.