Study Design Considerations for Estimation of Agreement with Contributions of Variability to Establish Reliability in a Medical Device Pivotal Study

• Activity Number: 687
• Type: Contributed
• Date/Time: Thursday, August 8, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Mat D. Davis*+ and Jeffrey L. Joseph
• Companies: Theorem Clinical Research and Theorem Clinical Research
• Keywords: Sensitivity ; Specificity ; Intraclass Correlation ; Kappa Statistic ; Reliability ; Medical Device

Abstract

For a diagnostic imaging agent, the major considerations for efficacy analysis in a medical device pivotal study are validity (sensitivity, specificity, agreement) and reproducibility of the result as used in clinical practice for diagnosis of disease or patient management. In order to have a successful clinical development program for regulatory approval, both results need to be prospectively demonstrated. Masked image evaluation by multiple independent readers is performed to provide information about the reproducibility. Sources of variation are misclassification (the agent result is inconsistent with the truth) and reader interpretation (each interprets the results with a certain sensitivity and specificity). Given the sensitivity and specificity of two readers, the intraclass correlation coefficient (ICC) and kappa will be determined. The effect of adding additional readers on the expected ICC will be demonstrated as well as the resulting difference if some readers interpret the results with poor sensitivity and specificity. The resulting bias and confidence interval width of the ICC estimator from increasing the number of subjects and readers will be presented.