SC03: Targeted Learning for Randomized Controlled Trials and Hybrid Studies in the Era of Artificial Intelligence and Machine Learning

Artificial intelligence (AI) and machine learning (ML) have the potential to revolutionize the medical product development and evaluation field, offering innovative solutions to accelerate processes, improve efficiency, and reduce costs. From identifying potential drug candidates, optimizing clinical trial designs to generating evidence, AI/ML technologies are transforming how therapies are discovered and brought to market. In recent years, the US Food and Drug Administration has recognized their potential and actively developed regulatory frameworks and guidance to ensure the safe and effective integration of AI/ML tools for drug development and evaluation. A recent landscape paper by FDA staff highlights the increasing number of submissions to FDA incorporating AI/ML components in the past few years, with even faster growth of AI/ML-driven submissions expected in the future. To address this need and stay ahead of the trend, statisticians must be prepared to understand and appropriately apply AI/ML methods that comply with regulatory requirements for planning, designing, and analyzing randomized controlled trials (RCTs) and real-world evidence (RWE) studies. This course will first provide an overview of FDA's perspectives on the use of AI/ML for drug development and evaluation, describing various uses of AI/ML across the full spectrum of drug development processes, landscape assessment of regulatory submissions, and available guidance and resources from FDA along with other FDA initiatives. Next, the course will introduce the targeted learning (TL) estimation roadmap for causal inference or both RCTs and RWE studies including hybrid design studies (i.e., RCT integrated with real-world data (RWD) sources such as data from routine clinical practice). The TL framework is particularly useful for guiding the appropriate use of AI/ML for regulatory purposes because of the following: (1) Study design and planning: TL offers step-by-step guidance to specifying key design components, including the target causal estimand, statistical estimand and identifying assumptions, required data, estimation, sensitivity analysis, and interpretation. (2) Efficient and flexible estimation approach: TL can leverage AI/ML at various design stages (e.g., power analysis, outcome identification, etc.) with guidance to safeguard statistical rigor and regulatory standards. In particular, it uses an efficient and flexible statistical method called targeted maximum likelihood estimation (TMLE) that can incorporate various ML algorithms to improve power in estimating the target effect without sacrificing the interpretability and validity of statistical inference while meeting regulatory recommendations. (3) Causal inference: TL is a particularly powerful tool for regulatory use, where understanding causal relationships is a priority. Existing papers on the TL roadmap offer a template for writing a statistical analysis plan, selecting the optimal combination of ML algorithms tailored to study characteristics, conducting sensitivity analyses and reporting the results. This course will describe how the TL roadmap and TMLE could be used in challenging areas of medical product development such as rare disease settings. Potential solutions to existing challenges will be presented through simulations and case studies demonstrating how TL and TMLE can improve design choices and enhance statistical power. The illustrations will include applications to both an RCT and RWE study.