A Novel Approach to Testing for Difference in Agreement Among Multiple Raters Between Two Measurement Techniques

• Activity Number: 193
• Type: Contributed
• Date/Time: Monday, August 10, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Biometrics Section
• Author(s): Huining Kang* and Ji-Hyun Lee and Yong Lin
• Companies: University of New Mexico and University of New Mexico and University of New Mexico
• Keywords: Observer agreement ; linear mixed-effect model ; power and sample size ; DLBCL patients

Abstract

In medical research we often need to compare the performances of two measurement techniques. When multiple raters (observers) are available, it is straightforward to calculate an agreement score for each technique, such as intraclass correlation coefficient (ICC) based on the observations of multiple raters obtained on the same subjects. However, the statistical inference for the difference of the scores for the two techniques has not been fully developed. We propose a likelihood ratio test based on linear mixed-effects model for testing the difference in the agreement among multiple raters between the two techniques. In addition, we provide a computational algorithm for power and sample size calculation of the method. We illustrate our method with a dataset from a study for immunohistochemical evaluation of MYC protein expression in diffuse large B-cell lymphoma (DLBCL) patients.