The Challenge in Making Inference About a Biomarker's Predictive Capacity

• Activity Number: 574
• Type: Invited
• Date/Time: Wednesday, August 12, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistics in Epidemiology
• Author(s): Holly Janes*
• Companies: Fred Hutchinson Cancer Research Center
• Keywords: biomarker ; prediction ; risk prediction ; risk model ; sensitivity and specificity ; non-regularity

Abstract

Biomarkers or patient characteristics that predict risk of an adverse outcome are highly sought after in many clinical contexts, and risk models are being developed for this purpose. Applications include guiding the use of therapy intended to prevent a disease, and selecting a therapeutic intervention for individuals diagnosed with a disease. A wide variety of statistical measures for characterizing the predictive capacity or performance of a risk model, and for contrasting the performance of different models, have been proposed. Often, the same data are used both to fit the risk model and to estimate its performance. In this setting, traditional approaches to doing inference about model performance, for example using normal theory or the bootstrap, do not perform well. We show that this is because the model performance estimators are all non-regular, and document the poor performance in simulations based on published studies. We also contrast measures of the performance of the fitted risk model and of the performance of the true risk model, and show that inference about either is problematic using traditional approaches. We discuss the practical implications of these result