Graphing and Modeling Kappa for Dependent Nominal Agreement Data with Application to Dental Caries

• Activity Number: 184
• Type: Contributed
• Date/Time: Monday, July 30, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistics in Epidemiology
• Author(s): Nancy Fan Cheng and William Santo and Stuart Gansky*+
• Companies: University of California at San Francisco and University of California at San Francisco and University of California at San Francisco School of Dentistry
• Keywords: Kappa ; Cluster ; Pair Chart ; Graphical ; Agreement ; Correlated

Abstract

In many epidemiologic studies, raters/examiners grade the disease outcome in multiple sites per person. Williamson et al. (2000) published methodology to estimate the kappa agreement statistic in nominal categorical data clustered within individuals, but the software to estimate cluster-adjusted kappas and their confidence intervals is not readily available. Bangdiwala (1987) conceived the agreement plot to extend Quade's pair chart (1973); Friendly (2000) developed a SAS macro to render the agreement plot. However, the agreement plot displays concordance on the off-diagonal, which is the opposite of the traditional contingency table layout with concordance on the diagonal. Thus, we developed SAS macros to estimate cluster-adjusted kappas & CIs for nominal disease outcomes and to graphically display agreement on the diagonal. We illustrate the methods with tooth surface calibration data from a dental caries trial. The agreement chart uses colors corresponding to traditional dental caries charts (e.g. red for decay, blue for fillings). Clinicians and researchers have found the revised agreement chart to be an intuitive explanation of agreement. Support: USDHHS/NIH/NIDCR U54DE019285.