Online Program Home
My Program

Abstract Details

Activity Number: 493
Type: Contributed
Date/Time: Wednesday, August 3, 2016 : 8:30 AM to 10:20 AM
Sponsor: Section on Risk Analysis
Abstract #320693 View Presentation
Title: Improvements in Estimation of Measures of Prediction Increment
Author(s): Danielle Enserro* and Ralph D'Agostino and Michael Pencina and Martin G. Larson
Companies: and Boston University and Duke University and Boston University
Keywords: NRI ; IDI ; confidence interval ; bias ; survival analysis ; bootstrapping
Abstract:

The net reclassification index (NRI) and integrated discrimination improvement (IDI) are statistics used to evaluate discrimination of risk prediction models. They supplement the area under the receiver operating characteristic curve (AUC). It is critical to estimate these measures precisely. In the survival analysis context, I study their empirical distributions and through bootstrapping I compare several methods to estimate confidence intervals. I perform a large simulation study using a Weibull survival distribution and bivariate normal risk factors. I compare performance under several scenarios: low and high event rates (10%, 50%), Type I and random censoring, and risk factor hazard ratios varying from null to strong. I use bootstrap resampling to estimate AUC, NRI, and IDI. I calculate confidence intervals for these statistics and evaluate bias and coverage probabilities. Finally, I make recommendations for proper use of bootstrapped confidence intervals based on best performance across these scenarios and apply the recommendations to real data in the first and second generation cohorts from the Framingham Heart Study.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

 
 
Copyright © American Statistical Association