Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 306 - SPEED: SPAAC SESSION II
Type: Topic-Contributed
Date/Time: Wednesday, August 11, 2021 : 3:30 PM to 5:20 PM
Sponsor: Section on Statistics in Defense and National Security
Abstract #318521
Title: Storytelling with Receiver Operating Characteristic (ROC) Curves for Environmental Remediation of Unexploded Ordnance (UXO)
Author(s): Shelley Cazares* and Jacob Bartel
Companies: Institute for Defense Analyses and Institute for Defense Analyses
Keywords: Environmental Remediation; Unexploded Ordnance (UXO); Receiver Operating Characteristic Curve (ROC Curve); Probability of Detection; Probability of False Alarm; False Alarm Rate

Receiver Operating Characteristic (ROC) curves are often used to assess the performance of binary classification systems, allowing stakeholders to understand tradeoffs between Type I (false positive) and Type II (false negative) errors. This works well in textbook cases. In real world experiments, however, ROC curves can have unexpected subtleties that make them difficult to construct and interpret. For example, the Department of Defense is sponsoring the development of novel sensors and software to identify Unexploded Ordnance (UXO) among clutter. UXO are duds—munitions that were previously armed and fired but failed to explode. UXO can still pose a risk of detonation even decades later, threatening the safety of nearby humans, animals, vegetation, and structures. Conducting blind tests to demonstrate finding UXO is fraught with safety, logistical, and cost constraints that make it difficult to construct the textbook ROC curves. Yet, with careful planning and a few key assumptions, ROC-like curves can still be crafted to quickly tell the story of how well novel sensors and software can detect, classify, and locate UXO versus clutter in terrestrial and underwater experiments.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2021 program