Safe Policy Learning Through Extrapolation: Application to Pre-Trial Risk Assessment

• Activity Number: 535 - Learning Individualized Treatment Rules in Complex Settings
• Type: Invited
• Date/Time: Thursday, August 11, 2022 : 10:30 AM to 12:20 PM
• Sponsor: Health Policy Statistics Section
• Author(s): Eli Ben-Michael* and Jim Greiner and Kosuke Imai and Zhichao Jiang
• Companies: Harvard University and Harvard University and Harvard University and University of Massachusetts Amherst
• Keywords: algorithm-assisted decision-making; observational studies; optimal policy learning; randomized experiments; robust optimization

Abstract

Algorithmic recommendations and decisions have become ubiquitous in today’s society. Many of these and other data-driven policies are based on known, deterministic rules to en- sure their transparency and interpretability. This is especially true when such policies are used for public policy decision-making. For example, algorithmic pre-trial risk assessments, which serve as our motivating application, provide relatively simple, deterministic classification scores and recommendations to help judges make release decisions. Unfortunately, existing methods for policy learning are not applicable because they require existing policies to be stochastic rather than deterministic. We develop a robust optimization approach that partially identifies the expected utility of a policy, and then finds an optimal policy by minimizing the worst-case regret. The resulting policy is conservative but has a statistical safety guarantee, allowing the policy-maker to limit the probability of producing a worse outcome than the existing policy. Lastly, we apply the proposed methodology to a unique field experiment on pre-trial risk assessments.