JSM 2015 Preliminary Program

Online Program Home
My Program

Abstract Details

Activity Number: 595
Type: Topic Contributed
Date/Time: Wednesday, August 12, 2015 : 2:00 PM to 3:50 PM
Sponsor: Biopharmaceutical Section
Abstract #316515
Title: Correcting for Over-Optimism in Metrics of Prognostic Model Improvement
Author(s): Megan Neely* and Michael Pencina
Companies: Duke University and Duke University
Keywords: prediction metrics ; nested risk models ; over-optimism bias ; bootstrapping
Abstract:

The use of prediction models is increasingly common in clinical research and practice.As new knowledge is gained the natural next step is to determine if adding the new information to existing models improves their performance. To address this question, a common approach is to compare the performance of the two nested models using one of many global or threshold-based prediction metrics. Researchers often report a model's apparent performance. This approach has been shown to result in overly optimistic estimates of a model's performance, but it is still chosen because it allows use of the entire data set when building the model. To address the issue, researchers often report bias-corrected estimates based on bootstrapping. However, the current approach for obtaining bias-corrected metrics does not provide estimates of precision. Further, the behavior of prediction metrics, apparent or bias-corrected, in the context of incremental model improvement has not been studied. This work will provide information about the performance of predictions metrics commonly used when comparing the prognostic performance of two nested models.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2015 program





For program information, contact the JSM Registration Department or phone (888) 231-3473.

For Professional Development information, contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

2015 JSM Online Program Home