Assessing the Goodness of Fit of Risk Prediction Rules in a Clustered Data Setting

• Activity Number: 458
• Type: Topic Contributed
• Date/Time: Wednesday, August 4, 2010 : 8:30 AM to 10:20 AM
• Sponsor: ENAR
• Author(s): Bernard Rosner*+ and Weiliang Qiu and MeiLing Ting Lee
• Companies: Harvard Medical School and Harvard Medical School and University of Maryland
• Keywords: Risk prediction ; ROC curves ; clustered data ; GEE

Abstract

The AUC is a commonly used metric to assess discrimination of risk prediction rules; however, standard errors of AUC are usually based on the Mann-Whitney U test that requires independence of sample units. For ophthalmologic applications, it is desirable to assess risk prediction rules based on eye-specific outcome variables which are generally highly , but not perfectly correlated in fellow eyes (eg. progression of individual eyes to AMD). In this article, we use the extended Mann-Whitney U test (Rosner et al, 2009) for the case where subunits within a cluster may have different progression status and assess goodness of fit of different prediction rules in this setting. Both data analyses based on progression of AMD and simulation studies show reasonable accuracy of this extended Mann-Whitney U test to assess the goodness of fit of eye-specific risk prediction rules.