Two Criteria for Evaluating Risk Prediction Models

• Activity Number: 46
• Type: Invited
• Date/Time: Sunday, July 31, 2011 : 4:00 PM to 5:50 PM
• Sponsor: ENAR
• Author(s): Ruth Pfeiffer*+ and Mitchell Gail
• Companies: National Cancer Institute and National Cancer Institute
• Keywords: Discriminatory accuracy ; Risk models ; Lorenz curve ; Discrimination

Abstract

We propose and study two criteria to assess the usefulness of models that predict risk of disease incidence for screening and prevention, or the usefulness of prognostic models for management following disease diagnosis. The first criterion, the proportion of cases followed PCF(q), is the proportion of individuals who will develop disease who are included in the proportion q of individuals in the population at highest risk. The second criterion is the proportion needed to follow-up, PNF(p), the proportion of the general population at highest risk that one needs to follow in order that a proportion p of those destined to become cases will be followed. PCF(q) assesses the effectiveness of a program that follows 100q% of the population at highest risk. PNF(p) assesses the feasibility of covering 100p% of cases by indicating how much of the population at highest risk must be followed. We show the relationship of PCF and PNF to the Lorenz curve and its inverse, and present distribution theory for their estimates. We develop methods for inference for a single risk model, and also for comparing the PCFs and PNFs of two risk models, both of which were evaluated in the same validation data