Evaluation of Predictive Model Performance of an Existing Model in the Presence of Missing Data

• Activity Number: 493 - Section on Risk Analysis CPapers 1
• Type: Contributed
• Date/Time: Thursday, August 6, 2020 : 10:00 AM to 2:00 PM
• Sponsor: Section on Risk Analysis
• Author(s): Pin Li* and Jeremy Taylor and Matthew Schipper
• Companies: University of Michigan and University of Michigan and University of Michigan
• Keywords: Area under the ROC curve; Brier score; Inverse probability weighting; Augmented inverse probability weighting; Multiple Imputation

Abstract

In medical research, the assessment of prediction models using data with missing covariate values is challenging. In this paper, we propose inverse probability weighted (IPW) and augmented inverse probability weighted (AIPW) estimates of the area under the ROC curve (AUC) and Brier Score to handle the missing data. We evaluated the performance of IPW and AIPW in comparison with multiple imputation (MI) in simulation studies under missing complete at random (MCAR), missing at random (MAR), and missing not at random (MNAR) scenarios. When there are missing observations in the data, MI and IPW can be used to obtain unbiased estimates of BS and AUC if the imputation model for the missing variable or the model for the missingness is correctly specified. MI is more efficient than IPW. AIPW can improve the efficiency of IPW, and also achieves double robustness from miss-specification of either the missingness model or the imputation model. The outcome variable should be included in the model for the missing variable under all scenarios, while it only needs to be included in missingness model if the missingness depends on the outcome.