Keywords: Artificial intelligence, Machine Learning, FDA, Regulation
State-of-the-art artificial intelligence (AI) systems now approach and sometimes exceed human levels of performance in many applications such as computer vision and speech recognition. Artificial intelligence (AI) algorithms for medical applications, e.g. Computer-Aided Detection (CADe) and Diagnosis (CADx) systems, have similarly seen a remarkable improvement in performance in last few years. This has led to a proliferation in development of AI devices beyond imaging applications to a variety of medical tasks using different input modalities. In this talk I will discuss some of the current principles used in the evaluation of AI devices in CDRH, along with various challenges that are unique to or more critical in medical devices compared to other use cases of AI. Examples of these challenges include curation of training/test datasets with associated ground truth, robustness and generalization, limited data set size, explainability/interpretability, and continuous learning. Finally, I will present examples of work from our laboratory at the Office of Science and Engineering Laboratories at CDRH which explore some of these themes as we prepare for regulating a wider variety of medical devices.