|
Abstract:
|
Multiple raters are often needed to be used interchangeably in practice for measurement or evaluation. Assessing agreement among these multiple raters via agreement indices are necessary before their participation. While the intuitively appealing agreement indices have been extended for assessing agreement among multiple raters, the extensions have limitations. The existing overall agreement indices either require normality and homogeneity assumptions or did not preserve the intuitive interpretation of the indices originally defined for two raters. In this paper, we propose a new set of overall agreement indices based on maximum pairwise differences among all raters. The proposed new indices retain the original intuitive interpretation from the pairwise version. Without making any distributional assumption, we also propose a new unified nonparametric estimation and inference approach for the overall indices based on generalized estimating equations that can accommodate replicates made by the same rater. Simulation studies are conducted to assess the performance of the proposed methods with and without replications.
|
