All Times EDT
Keywords: Inpatient Quality Indicator, Machine Learning, Predictive Modeling, Model Comparison, Classification, Random Effect, LightGBM
This study added a random effect to risk prediction models trained with logistic regression and gradient boosting to see whether accounting for clustering by hospital improved prediction of AHRQ inpatient quality indicators (IQIs). Primary outcomes were indicators of acute myocardial infarction mortality (IQI-15) and acute stroke mortality (IQI-17). Hospital admission data were used from the 2016 National Inpatient Sample. Features such as patients’ demographic, hospital information, comorbidity indicators and chronic condition indicators were either included in the NIS dataset or created by a set of HCUP tools and software. After feature engineering, 96,933 admissions with 63 features were included for IQI-15 (5.86% with the event) and 65,038 admissions with 64 features were included for IQI-17 (13.05% with the event). A random hospital effect was added to the LightGBM model by fitting a generalized linear mixed model with a random hospital intercept and one fixed effect – the predicted probability from LightGBM. Test AUC was estimated via 10-fold cross validation on known hospitals and new hospitals (based on whether these hospitals were included in the training folds). We found that the LightGBM AUCs (IQI-15 known: 0.844 (SE: 0.0064); IQI-15 new: 0.844 (SE: 0.0023); IQI-17 known: 0.809 (SE: 0.0090); IQI-17 new: 0.808 (SE: 0.0072)) were higher than logistic models (IQI-15 known: 0.835 (SE: 0.0062); IQI-15 new: 0.835 (SE: 0.0088); IQI-17 known: 0.788 (SE: 0.0081); IQI-17 new: 0.787 (SE: 0.008)) for predicting both quality indicators. When predicting for known hospitals, models with a random hospital effect were slightly better (except LightGBM on IQI-15), but only slight differences were found. It is possible that in datasets with greater random effect variation, models that account for clustering would show greater improvement in prediction.