Statistical Considerations When Combining Artificial Intelligence–Enabled Diagnostic Devices
Manasi Sheth
Food and Drug Administration
Daniel Erchul
US Food and Drug Administration
Gene Pennello
US Food and Drug Administration
Artificial intelligence (AI) is increasingly being incorporated into medical devices in efforts to provide results used alone or with other information to help assess a subject's present or future state of health. Increased computing power, greater availability of data, and the availability of deep learning and other AI methods have opened many opportunities for developing AI-enabled medical devices with increased capabilities. Performance validation of AI-enabled medical devices is essential in order to assess safety and efficacy. We focus on statistical methods to assess the combination of two or more AI diagnostic devices in order to improve diagnostic accuracy - e.g., sensitivity, specificity, positive and negative predictive value, diagnostic likelihood ratio. After providing a brief discussion of AI machine learning, including convolutional neural networks with application to diagnostic medical imaging, we present some basic probabilistic considerations and statistical techniques regarding combination of two binary-output AI-enabled devices.