.png)
632 – Theories and Applications of Paradata in a Mixed Mode Data Collection
Study Design Considerations for Estimation of Agreement with Contributions of Variability to Establish Reliability in a Medical Device Pivotal Study
Mat D. Davis
Theorem Clinical Research
Jeffrey L. Joseph
Theorem Clinical Research
For a diagnostic imaging agent, the major considerations for efficacy analysis in a medical device pivotal study are validity (sensitivity, specificity, agreement) and reproducibility of the result as used in clinical practice for diagnosis of disease or patient management. In order to have a successful clinical development program for regulatory approval, both results need to be prospectively demonstrated. Masked image evaluation by multiple independent readers is performed to provide information about the reproducibility. Sources of variation are misclassification (the agent result is inconsistent with the truth) and reader interpretation (each interprets the results with a certain sensitivity and specificity). Given the sensitivity and specificity of two readers, the intraclass correlation coefficient (ICC) and kappa will be determined. The effect of adding additional readers on the expected ICC will be demonstrated as well as the resulting difference if some readers interpret the results with poor sensitivity and specificity. The resulting bias and confidence interval width of the ICC estimator from increasing the number of subjects and readers will be presented.