Online Program

Equitable Prediction of Suicide from Administrative Patient Records (309719)

Harish Subrahmanya Bhat, University of California, Merced 
Sidra Goldman-Mellor, University of California, Merced 
*Majerle Reeves, University of California, Merced 

Keywords: equitability, suicide, imbalanced data, classification

When machine learning models use sensitive attributes (such as racial and/or ethnic identity) as predictors, it is important to pay particular attention to how the data are preprocessed. Here we focus on building predictive models for suicide death from a database of administrative patient records linked with death records. We treat each visit by each patient as an independent record. Our data set consists of more than 44 million records, of which just over 0.09% correspond to eventual death by suicide. Hence we have a severely imbalanced classification problem. A common approach for such problems is to bootstrap (i.e., sample with replacement from) the smaller class to create a balanced data set for the purposes of training. Here we show that this method results in classifiers with poor performance on non-White ethnic/racial groups. We also show that strict removal of sensitive features (and their proxies) can hurt classification accuracy for all groups, even if they do increase equitability across groups.

To remedy this, we propose two preprocessing methods, (i) building separate models by sensitive class and (ii) equity-directed bootstrapping. Equity-directed bootstrapping creates a training set that is balanced with respect to both ethnic/racial groups and class labels. We compare four predictive modeling techniques combined with four preprocessing methods, including (i) and (ii). For gradient boosted tree models, we see the standard deviation of the test set sensitivity (respectively, test set specificity) across racial/ethnic groups go from 0.15 (resp., 0.20) in the model with no special considerations to 0.01 (resp., 0.02) in the separately trained models and 0.02 (resp., 0.03) in the model trained with equity-directed bootstrapping. In this context, smaller standard deviations imply more equitable predictions across groups. Overall, methods (i) and (ii) dramatically improve the test set sensitivity of the classifiers across minority groups.