A Comparison of Confidence/Credible Interval Methods for the Area Under the ROC Curve for Continuous Diagnostic Tests with Small Sample Size

• Activity Number: 505
• Type: Contributed
• Date/Time: Wednesday, August 12, 2015 : 8:30 AM to 10:20 AM
• Sponsor: Biopharmaceutical Section
• Author(s): Dai Feng* and Giuliana Cortese and Richard Baumgartner
• Companies: Merck Research Laboratories and University of Padova and Merck
• Keywords: AUC ; Small sample size ; Mann-Whitney ; Higher order asymptotic; ; Behrens-Fisher problem ; Bayesian MCMC

Abstract

The area under the ROC curve (AUC) is arguably the most widely used summary index for the ROC curve. In this paper, we describe and compare twenty-nine non-parametric and parametric methods for the construction of the CI for the AUC when the number of available observations is small. The methods considered include not only those that have been widely adopted but also those that have been less frequently mentioned or, to our knowledge, never applied to the AUC context. We found by a comprehensive simulation study that the larger the true AUC value and the smaller the sample size, the large the discrepancy among the results of different approaches. When the model is correctly specified, the parametric approaches tend to outperform the non-parametric ones. Moreover, in the non-parametric domain, we found that a method based on the Mann-Whitney statistic is in general superior to the others. We further elucidate potential issues and provide possible solutions to along with general guidance on the CI construction for the AUC when the sample size is small. Finally, we illustrate the utility of different methods through real life examples.