|
Abstract:
|
In the context of survival analysis models, the Brier score and C-statistics have often been used for calibration and discrimination purposes, respectively. This presentation will discuss important properties of these two model performance metrics that are overlooked in the literature. Using extensive simulation, the dependence of the Brier Score on the time horizon in our study, event rate and other model parameters will be examined closely. Similarly, the performance of the C-statistic as a function of number of events, sample size, number of predictors, noise variables and other model parameters will be presented in detail. Special focus is given to the finite sample performance of these statistics and the problem of model overfitting. We caution against measuring predictive accuracy using these statistics without considering the setting from which our data may have been obtained. For example, a very low Brier score can be obtained under a setting where all the predictors have no association with the outcome. Application data from the field of cardiology will be utilized to illustrate important properties of these two model performance measures.
|
