Online Program Home
My Program

Abstract Details

Activity Number: 555 - Using Surveys to Improve the Representativeness of Nonprobability Samples in Epidemiologic Studies
Type: Invited
Date/Time: Wednesday, August 1, 2018 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistics in Epidemiology
Abstract #326668 Presentation
Title: Evaluating Disease Prediction Models Using a Cohort Whose Covariate Distribution Differs from That of the Target Population
Author(s): Alice S Whittemore*
Companies: Stanford University
Keywords: Cohort selection bias; calibration; concordance; personal predictive model; weighted-as-needed

Personal predictive models for adverse outcomes are important in medicine. Model performance is evaluated by application to the covariates of participants in cohort studies, with model predictions compared to subjects' subsequent outcome incidence. However the covariates of cohort participants seldom represent those of a model's target population. Since performance estimates depend on the covariate distribution in the evaluation sample, covariate differences between cohort and target populations can cause misleading estimates of model performance in the target population. We address this problem by weighting the cohort subjects to make their covariate distribution better match that of the target population. We show that the method provides accurate estimates of model performance in the target population, while un-weighted estimates may not. We illustrate the method by applying it to evaluate an ovarian cancer prediction model targeted to US women, using cohort data from participants in the California Teachers Study. The methods can be implemented in the R-package RMAP (Risk Model Assessment Package) available at

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program