Statistical Consideration in Demonstrating Standalone Performance for AI/ML-Based Computer-Aided Detection Devices

• Activity Number: 378 - Study Design and Statistical Challenges for AI/ML Based Medical Tests
• Type: Topic-Contributed
• Date/Time: Thursday, August 12, 2021 : 12:00 PM to 1:50 PM
• Sponsor: Section on Medical Devices and Diagnostics
• Author(s): Jessie Moon*
• Companies: FDA
• Keywords: Artificial intelligence; Machine learning; Standalone performance; Radiology device; Computer-assisted detection; Multiple abnormalities

Abstract

AI/ML based techniques, such as neural networks, deep learning etc., are being used in computer-assisted detection devices applied to radiology images and radiology device data. Some applications include devices for detecting single abnormality and devices which utilize single or multiple algorithms to detect multiple abnormalities. For computer-assisted detection devices that do not require reader's (physician or other health care professional) to interpret the device output, a standalone study is usually conducted for marketing purposes. This presentation will describe some statistical challenges in the design and analysis of studies for validating standalone performance, especially for devices that are intended to detect multiple abnormalities. Using case studies of recently cleared devices, we will discuss FDA reviewers perspective on study design considerations for assessing standalone performance for different indications.