Kappa Statistic for Surveys and Complex Data

• Activity Number: 521
• Type: Contributed
• Date/Time: Wednesday, August 1, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Section on Survey Research Methods
• Author(s): G. Gordon Brown*+ and Gayle S Bieler and Robert J Morris
• Companies: RTI International and RTI International and RTI International
• Keywords: Kappa ; Survey ; SUDAAN

Abstract

In this talk we present a method for calculating the survey-weighted Kappa measure of agreement statistic and producing an appropriate variance and test of significance from complex data that may include clustering, weighting, stratification, and other features of sample surveys. The variance can be calculated using a variety of methods, including implicit Taylor series linearization and replication methods. Recent papers by Zajacova and Dowd (2011) and Winn, Clifford, Johnson and Kingman (1999) illustrate the growing popularity and need for survey-weighted Kappa in the public health setting. We use SUDAAN 11 to illustrate the survey Kappa in two examples discussed by these authors: 1) National Health and Nutrition Examination Survey (NHANES) III survey, estimating the Kappa measure of intra-dental examiner agreement between ratings of baseline vs. follow-up periodontal assessments of teeth; and 2) 2005-2008 continuous NHANES, estimating agreement between two measures of self-reported health status.