An Extension of Cohen’s Kappa for Clustered Data and Group Sequential Testing

• Activity Number: 170 - SPEED: Biopharmaceutical Methods and Application I, Part 1
• Type: Contributed
• Date/Time: Monday, July 29, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Mary Ryan* and Daniel L. Gillen
• Companies: University of California, Irvine and University of California, Irvine
• Keywords: Cohen's Kappa; Group sequential testing; Clustered data; Multiple outputation

Abstract

We present methodology motivated by a controlled trial designed to validate SPOT GRADE, a novel surgical bleeding severity score (Spotnitz et al, Spine, 2018). Briefly, the study was designed to quantify inter- and intra-surgeon agreement for characterizing the severity of surgical bleeds via a Kappa statistic. Multiple surgeons were presented with a randomized sequence of controlled bleeding videos and asked to apply the rating system to characterize each wound. Each video was presented multiple times in a randomized fashion, resulting in clustered data. In this work we implement a multiple outputation procedure to account for within-video clustering and embed the testing procedure in a group sequential framework to increase study efficiency. We establish independent increments for the proposed multiple outputation-based Kappa statistic, allowing for the application of standard group sequential stopping boundaries and monitoring procedures. Operating characteristics of the proposed method are assessed via simulation and applied to data from the SPOT GRADE trial. We illustrate potential sample size savings relative to a fixed sample design and consider trade-offs with power.