Transportation of Area Under the ROC Curve to a Target Population

• Activity Number: 231 - SPEED: SPAAC SESSION I
• Type: Topic-Contributed
• Date/Time: Wednesday, August 11, 2021 : 10:00 AM to 11:50 AM
• Sponsor: Biometrics Section
• Author(s): Bing Li* and Constantine Gatsonis and Issa J. Dahabreh and Jon Steingrimsson
• Companies: Brown University and Brown University and Harvard T. H. Chan School of Public Health and Brown University
• Keywords: Domain adaptation; covariate shift; generalizability; U-processes; model performance; importance weighting

Abstract

We develop methods for estimating the area under the ROC curve (AUC) of a prediction model in a target population that differs from the source population population used for original model development. We focus on the setting where outcome and covariate data are available from the source population, but only covariate data are available from the target population. If covariates that affect model AUC are differently distributed between the source and target population, AUC estimators that only use data from the source population are biased for the target population AUC. We provide identifiability conditions and results under which the target population AUC is identifiable. We develop three estimators for the target population AUC and show that they are consistent and asymptotically normal. We evaluate their finite-sample performance using simulations and we apply them to estimate the AUC of a lung cancer risk prediction model using source population data from the National Lung Screening Trial (NLST) and target population data from the National Health and Nutrition Examination Survey (NHANES).