Increasing the Transparency of Black-Box Systems

• Activity Number: 27 - SDNS Speed Session
• Type: Contributed
• Date/Time: Sunday, August 8, 2021 : 1:30 PM to 3:20 PM
• Sponsor: Section on Statistics in Defense and National Security
• Author(s): Cami Fuglsby* and Christopher Saunders and Danica M Ommen and JoAnn Buscaglia and Michael Caligiuri
• Companies: South Dakota State University and South Dakota State University and Iowa State University and FBI Laboratory and University of California San Diego
• Keywords: automated handwriting system; pairwise scores; degenerate U-statistics; black-box system; white-box system; interpretable artificial intelligence

Abstract

Though often highly accurate, computational algorithms used in proprietary forensic-biometric identification systems are opaque and, therefore, pose a challenge for proper discovery in the U.S. judicial system. To increase transparency and interpretability for discovery, many have called for the release of the source code, potentially infringing the intellectual property of the algorithm developers.

In this work, we propose a middle ground between access to intellectual property and the need for interpretability of said algorithms. Our approach develops techniques to characterize the performance of an ‘opaque’ black-box algorithm in terms of a ‘clear’ white-box algorithm. This work emerged from an analysis of handwriting samples that were processed through two automated feature extraction systems: one clear and one opaque. The features from pairs of writing samples were compared using two scoring algorithms. Our goal is to develop a significance test for the null hypothesis that a ‘clear’ score is unrelated to the ‘opaque’ score. We develop strategies for estimating a response surface to characterize the black-box algorithm in terms of the white-box algorithm.