|
Abstract:
|
In view of improving breast cancer detection rates, large scale agreement studies have been conducted to evaluate the consistency of cancer ratings of many radiologists. Challenges arise in these studies where a sample of patients undergo two screening tests, resulting in a complex correlation structure between experts' ratings. Here we propose a novel paired kappa measure to compare the agreement between the binary ratings of many radiologists across two cancer screening tests. The proposed kappa appropriately accounts for dependencies between a patient's ratings, corrects for agreement due to chance and is robust to disease prevalence and other flaws often observed in the use of Cohen's kappa. In contrast to existing approaches, the proposed measure can flexibly incorporate large numbers of experts and patients by utilizing the generalized linear mixed models framework. Methods are applied to a recent nationwide breast cancer screening study comparing the ratings of film to digital mammography.
|
