|
Abstract:
|
The net reclassification index (NRI) and integrated discrimination improvement (IDI) are statistics used to evaluate discrimination of risk prediction models. They supplement the area under the receiver operating characteristic curve (AUC). It is critical to estimate these measures precisely. In the survival analysis context, I study their empirical distributions and through bootstrapping I compare several methods to estimate confidence intervals. I perform a large simulation study using a Weibull survival distribution and bivariate normal risk factors. I compare performance under several scenarios: low and high event rates (10%, 50%), Type I and random censoring, and risk factor hazard ratios varying from null to strong. I use bootstrap resampling to estimate AUC, NRI, and IDI. I calculate confidence intervals for these statistics and evaluate bias and coverage probabilities. Finally, I make recommendations for proper use of bootstrapped confidence intervals based on best performance across these scenarios and apply the recommendations to real data in the first and second generation cohorts from the Framingham Heart Study.
|
