Modifying Artificial Intelligence/Machine Learning-Based Software and the Algorithm Change Protocol

• Activity Number: 400 - Challenges and Opportunities for the Principled Calibration and QA/QC Assessments of AI and Machine Learning Methods Within Medical Device Applications
• Type: Invited
• Date/Time: Wednesday, August 10, 2022 : 10:30 AM to 12:20 PM
• Sponsor: Section on Medical Devices and Diagnostics
• Author(s): Tarek Haddad* and Donald Musgrove
• Companies: Medtronic Inc. and Medtronic Inc.
• Keywords: AI; Machine learning; regulatory

Abstract

Regulation of AI-based Software as a Medical Device is slowly growing. There are several challenges around algorithm updates, including the algorithm change protocol (ACP) and the predetermined change control plan (PCCP). Algorithm changes are not expected to fall under one of the traditional approval pathways, e.g., 510(k), De Novo Classification, or a PMA, and the regulatory burden of algorithm changes will be related to risk, i.e., the patient condition and the significance of information provided by the algorithm. Algorithm changes of interest include model performance, algorithm inputs, and the intended use. Drawing from our experience with a recent submission and approval, we highlight several algorithm changes that do and do not require a premarket review. These changes include new model architecture, training with new data, modifications to the modeling input, and expansion to new use cases. We also highlight future challenges, including the concept of Good Machine Learning Practices and real-world performance monitoring.