Unadjusted and Adjusted Receiver Operating Characteristic Curve and Precision Recall Curve Analyses for Complex Survey Data

• Activity Number: 352 - Small Area Estimation, Analysis of Complex Sample Survey Data, and New Advances for Health Surveys
• Type: Contributed
• Date/Time: Thursday, August 12, 2021 : 10:00 AM to 11:50 AM
• Sponsor: Survey Research Methods Section
• Author(s): Alok Kumar Dwivedi*
• Companies: Texas Tech University Health Sciences Center El Paso
• Keywords: Receiver operating characteristic curve ; Precision-recall curve ; Survey data analysis; Adjusted area under the curve; Predictive study ; Diagnostic study

Abstract

Receiver operating characteristic curve (ROC) analysis is often conducted to evaluate the prediction and diagnostic accuracy of a binary outcome. For imbalanced datasets, a precision-recall curve (PRC) is more useful than the ROC analysis. The sampling weights need to be included in developing predictive models using complex survey datasets. We developed STATA modules for performing ROC and PRC analyses with area under the curve (AUC) computations for complex survey studies. We further developed adjusted ROC and PRC methods for evaluating quantitative classifiers after adjusting for predictors. We used the National Health and Nutrition Examination Survey data for evaluating the performance of sex hormone binding globulin (SHBG) for predicting metabolically healthy status in adult females. The AUC was estimated as 67% using ROC and 49% using PRC analyses without survey weights. With survey weight parameters, the AUC was estimated to be 70% using ROC and 54% using PRC analyses. The age-adjusted AUC of SHBG was estimated as 72% using ROC and 66% with PRC analyses. Our proposed methods have potential applications in predictive and diagnostic studies using complex survey methods.