|
Abstract:
|
In medical and social science research, analyzing inter or intra-observer agreement provides a useful means of assessing the reliability of a rating system. Such assessments are particularly relevant in radiology, biomarker, and survey research among others, as they play a critical role in disease diagnostics and prognosis. Often comparison of agreement across multiple testing methods is sought where testing is carried out on the same experimental units rendering the outcomes to be correlated. In this talk, we present a Bayesian method for comparing dependent agreement measures. Simulation studies showed that the proposed methodology outperforms the competing frequentist methods in terms of type I error and power even when the rating probabilities differ moderately. We further developed a Bayesian model for comparing dependent agreement measures adjusting for subject- and rater-level heterogeneity. We adopted a joint analysis that alleviates potential bias stemming from the two-stage method. The developed methodology was implemented on the findings from a study evaluating classification methods for chest radiographs for pneumoconiosis developed by the International Labor Office.
|
