186 – Biomarker and Diagnostic Test Evaluation
Evaluating MRMC Studies with Binary Endpoints
Changhong Song
FDA
Multi-Reader Multi-Case (MRMC) study designs have been used to evaluate the diagnostic performance of medical devices or procedures in different areas such as radiological imaging and digital pathology. The primary statistical evaluations for MRMC studies have commonly been based on receiver operating characteristic (ROC) curve with observer performance evaluation paradigm using area under the curve (AUC). However, for many diagnostic clinical studies, readers only provide binary responses and therefore the primary performance metrics are sensitivity and specificity, instead of AUC of ROC. Most literature for MRMC studies focus on the AUC analysis. There are few systematic evaluations and comparisons of the various methods for MRMC studies with binary endpoints. In this study, we summarize and evaluate how some commonly used statistical methods for MRMC studies can be used to evaluate studies with binary study outputs.