Transportability of Risk Prediction Models

• Activity Number: 268 - Replicability and the Narrative of Scientific Research
• Type: Invited
• Date/Time: Wednesday, August 11, 2021 : 1:30 PM to 3:20 PM
• Sponsor: American Association for the Advancement of Science
• Author(s): Jon Steingrimsson* and Constantine Gatsonis and Issa Dahabreh and Bing Li and Samantha Morrison
• Companies: Brown University and Brown University and Brown University and Brown University and Brown University
• Keywords: transportability; covariate shift; prediction model; generalizability; domain adaptation; model performance

Abstract

Prediction models are often used or interpreted in the context of a target population that differs from the study population used to develop the model (e.g., a different health-care system or a different geographic region). When the distribution of prediction error modifiers differs between the target population and the study population, naively assuming that properties of the study-based prediction model transport to the target population can lead to bias. In this talk, we assume that outcome and covariate information is available from the study data and covariate but no outcome information is available on a sample from the target population. We provide conditions under which measures of model performance in the target population are identifiable using the observed data and we develop and discuss properties of three estimation procedures - inverse probability weighting, outcome model, and doubly robust estimators. Finite sample performance is evaluated using simulations and using data on cancer patients.