Inference for the Area Under the Curve (AUC) in a Three-Level Clustered Data Setting

• Activity Number: 337 - Approaches for Modeling Clustered and Longitudinal Data
• Type: Contributed
• Date/Time: Wednesday, August 5, 2020 : 10:00 AM to 2:00 PM
• Sponsor: Biometrics Section
• Author(s): Camden Bay* and Bernard Rosner and Robert Glynn
• Companies: Brigham and Women's Hospital and Brigham and Women's Hospital and Brigham and Women's Hospital
• Keywords: auc; clustered data; correlated data; three-level; wilcoxon-mann-whitney

Abstract

The area under the empirical ROC curve (AUC) is routinely used to determine how strongly a given model discriminates between the levels of a binary outcome. Rosner, Qiu, and Lee proposed methods for estimation of the variance of the AUC in the setting of two-level clustered data using probit-transformed results [Lifetime Data Analysis, 19(2): 242-256 (2013)]. We have extended this approach so that inference for the AUC may be performed in a three-level clustered data setting. The performance of 95% confidence intervals around the observed AUC was assessed through simulation of three-level clustered data in multiple settings. Across all settings, the actual confidence interval coverage varied from 0.933 to 0.955 and the ratio of the calculated to the actual variance of the AUC varied from 0.834 to 1.085. In addition to simulations, we used this method to assess predictors of hypertension in a dataset from South Wales. The dataset included 966 adults ? 30 years old in 318 families, each person having 1-4 measurements over a 10-15 year period. The AUC, estimated using a validation set, was 0.73 (95% confidence interval 0.67 to 0.78).