Uncertainty in Estimation of ROC Curves with Applications to Forensic Science

• Activity Number: 190 - Synergy of Bayesian, Frequentist, and Fiducial Approaches in Addressing Modern Statistical Problems in Data Science
• Type: Invited
• Date/Time: Tuesday, August 10, 2021 : 1:30 PM to 3:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Jan Hannig* and Hary Iyer
• Companies: University of North Carolina at Chapel Hill and National Institute of Standards and Technology
• Keywords: generalized fiducial inference; Receiver Operating Characteristic; Area Under Curve; Forensic statistics

Abstract

Receiver operating characteristic (ROC) curve is an extremely useful tool for examining the performance of a two-class classifier as well as for comparing two or more such classifiers. ROC plots or curves are widely used not only for comparing machine learning classifiers but also in many other disciplines including medical diagnostic testing and forensic evidence assessment. There are a variety of summary statistics arising from ROC curves – sensitivity, specificity, positive predictive value (PPV) , negative predictive value (NPV), area under the curve (AUC), and so on. The likelihood ratio summarizing the strength of evidence in favor of one class versus the other class is the slope of the ROC curve corresponding to the evidential observations. For these reasons, it is desirable to have well performing procedures for inferences about ROC curves and its summary statistics. In this talk we will discuss generalized fiducial inference for these quantities and compare its performance with existing frequentist and Bayesian methods.