JSM 2013 Home
Online Program Home
My Program

Abstract Details

Activity Number: 145
Type: Contributed
Date/Time: Monday, August 5, 2013 : 8:30 AM to 10:20 AM
Sponsor: Section on Statistics in Epidemiology
Abstract - #309765
Title: Combined Statistical Approaches for Comparing Performances of Two Independent Prediction Models
Author(s): Hui Zhou*+ and Jeff M Slezak and Stephen F. Derose and Don Morris and Anny H Xiang and Steve J. Jacobsen
Companies: Kaiser Permanente and Southern California Permanante Medical Group and Southern California Permanente Medical Group and Archimedes and Kaiser Permanente and Southern California Permanente Medical Group
Keywords: Prediction model ; AUC ; predictiveness curve
Abstract:

The importance of risk prediction models is well recognized and it is critical to evaluate the models' discrimination and calibration performance. Area under ROC curve (AUC) is used to measure discrimination. Hosmer-Lemeshow test measures how well the model can correctly predict events. A predictiveness curve provides a visual representation of fit without quantitative information. In this study, we combined three approaches to compare two independent models performance. Predictiveness curves were produced for both models, comparing the observed event proportion with average estimated risk in each predicted risk decile. Hosmer-Lemeshow (H value) and AUC were applied to provide statistical parameters such as p-value. Simple bootstrap technique was used to create 1000 subsets by randomly selecting samples with replacement from the original data set. In each subset, the H-value and AUC from two models were compared. The percent of times when one model was superior to another among the 1000 datasets was used to obtain a p-value. Combining the three statistical methods provided visual and qualitative information for two models comparison, which was relevance to clinic practice.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2013 program




2013 JSM Online Program Home

For information, contact jsm@amstat.org or phone (888) 231-3473.

If you have questions about the Continuing Education program, please contact the Education Department.

The views expressed here are those of the individual authors and not necessarily those of the JSM sponsors, their officers, or their staff.

ASA Meetings Department  •  732 North Washington Street, Alexandria, VA 22314  •  (703) 684-1221  •  meetings@amstat.org
Copyright © American Statistical Association.