|
Abstract:
|
Risk Assessment Instruments (RAIs) are predictive tools used to aid decision making in domains such as criminal justice and child welfare. RAIs estimate the risk of an adverse outcome, such as child neglect; this estimated risk may then inform a decision such as whether to screen in a child welfare case for investigation. RAIs are naturally concerned with the potential outcomes associated with available decisions, such as what would happen to a child if no investigation were initiated. This means that, contrary to current practice, both the predictive performance and the fairness properties of RAIs should be evaluated with respect to potential outcomes, rather than historical observed outcomes (Coston et al., 2020). Here, we develop novel methods to post-process arbitrary existing predictors to render them (1) loss-optimal with respect to potential outcomes and (2) fair in the sense that they satisfy (approximate) counterfactual equalized odds (Hardt et al., 2016; Coston et al., 2020). We derive estimators and show both theoretically and via simulations that they are consistent at fast rates.
|
