|
Activity Number:
|
310
- Data Integration and Information Synthesis in Survival Analysis
|
|
Type:
|
Invited
|
|
Date/Time:
|
Tuesday, August 9, 2022 : 2:00 PM to 3:50 PM
|
|
Sponsor:
|
Lifetime Data Science Section
|
|
Abstract #320466
|
|
|
Title:
|
Accommodating Population Differences in Model Validation
|
|
Author(s):
|
Ruth Pfeiffer* and Yiyao Chen and Mitchell Gail and Donna P. Ankerst
|
|
Companies:
|
National Cancer Institute and Technical University of Munich and National Cancer Institute and Technical University of Munich
|
|
Keywords:
|
Calibration;
Discrimination;
Selection;
Verification;
Performance assessment
|
|
Abstract:
|
Validation of risk prediction models in independent data provides a rigorous assessment of model performance. However, several differences between the populations that gave rise to the training and the validation data can lead to seemingly poor performance of a risk model. We formalize the notions of similarity of the training and validation data and define reproducibility and transportability. We address the impact of different predictor distributions and differences in verifying the outcome on model calibration, accuracy and discrimination. When individual level data from both the training and validation data sets are available, we propose and study weighted versions of the validation metrics that adjust for differences in the predictor distributions and in outcome verification to provide a more comprehensive assessment of model performance. We give conditions on the model and the training and validation populations that ensure a model's reproducibility or transportability and show how to check them. We discuss approaches to recalibrate a model. As an illustration we develop and validate a prostate cancer risk model using data from two large prostate cancer screening trials.
|
Authors who are presenting talks have a * after their name.
