Pearsonian Estimation and Inference for a Model-Based Measure of Agreement

• Activity Number: 253
• Type: Contributed
• Date/Time: Monday, July 30, 2012 : 2:00 PM to 3:50 PM
• Sponsor: Biometrics Section
• Author(s): Jie Gao*+ and Donald George Edwards and Andrew Wood and Jayaram Sethuraman
• Companies: University of South Carolina and University of South Carolina and University of Nottingham and Florida State University
• Keywords: Pigeonhole bootstrap ; U-statistics ; generalized method of moments

Abstract

Cohen's ? (Educ. Psychol. Meas, 1960) is widely used for the assessment of agreement among raters each classifying subjects as diseased or not. Nelson and Edwards (Can. J. Stat., 2008) propose a generalized linear mixed model for the agreement process, showing that Cohen's ? can seriously underestimate agreement under this model and proposing a model-based coefficient ?m. Our paper adds theoretical justification and practical methods for estimation and inference for ?m. The Nelson-Edwards model for the agreement process is motivated by a threshold model with latent crossed random effects, item-rater interaction, and pure error. Under this structure ?m?is an increasing function of Pearson's (Philos. Trans. Roy. Soc. A, 1900) tetrachoric correlation between the latent effects of any two raters. A practical method for estimation and inference on ?m, analogous to that used by Pearson, is proposed. Asymptotics are established using a generalization of Hoeffding's (Ann. Math. Statist. 1948) method. The new methods are shown to be reliable at moderate sample sizes using simulation. A mammography example (Beam et al, Arch. Intern. Med.,1996) shows substantial differences between ?m and ?.