Comparing Diagnostic Accuracies in a Multi-Reader, Multi-Test Design

• Activity Number: 26
• Type: Topic Contributed
• Date/Time: Sunday, August 4, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistics in Imaging
• Author(s): Eunhee Kim*+ and Donglin Zeng and Zheng Zhang
• Companies: Brown University and The University of North Carolina and Brown University
• Keywords: Area under the ROC curve ; receiver operating characteristic curve ; multi-reader, multi-test design ; U-statistics

Abstract

ROC analysis is widely used to evaluate the performance of diagnostic tests with continuous or ordinal responses. A popular study design for assessing the accuracy of diagnostic tests involves multiple readers interpreting multiple diagnostic test results; that is called the multi-reader, multi-test design. In this paper, we propose a nonparametric approach to estimate and compare the accuracies of diagnostic tests using this design. We extend DeLong et al.'s approach to accommodate multi-reader, multi-test data and use the general theory of U-statistics for hypothesis testing and inference. Moreover, we propose a formula for power calculation based on the asymptotic normality of the area under the ROC curves. We compare our approach to the Obuchowski-Rockette (OR) method via simulation studies and a real data analysis. The simulation studies demonstrate that the proposed method performs well even when the number of readers is small compared to the OR procedure.