Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 500 - Statistical Learning
Type: Contributed
Date/Time: Thursday, August 6, 2020 : 10:00 AM to 2:00 PM
Sponsor: Section on Statistical Learning and Data Science
Abstract #313344
Title: When Black Box Algorithms Are (Not) Appropriate: A Principled Prediction-Problem Ontology
Author(s): Jordan Rodu* and Michael Baiocchi
Companies: University of Virginia and Stanford University
Keywords: Machine Learning; Black-box algorithm; Prediction; Accountability; Fairness; Common Task Framework

In the 1980s a new, extraordinarily productive way of reasoning about algorithms emerged. Though this type of reasoning has come to dominate areas of data science, it has been under-discussed and its impact under-appreciated. For example, it is the primary way we reason about ``black box'' algorithms. In this paper we analyze its current use (i.e., as ``the common task framework'') and its limitations; we find a large class of prediction-problems are inappropriate for this type of reasoning. Further, we find the common task framework does not provide a foundation for the deployment of an algorithm in a real world situation. Building off of its core features, we identify a class of problems where this new form of reasoning can be used in deployment. We purposefully develop a novel framework so both technical and non-technical people can discuss and identify key features of their prediction problem and whether or not it is suitable for this new kind of reasoning.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2020 program