PS33: Computing, Optimization, and Causal Inference for Adaptive Clinical Trials

The implementation of Bayesian adaptive, causal inference, and artificial intelligence (AI) tools in clinical trials presents unique practical challenges, particularly when integrating these methodologies into software products. Commercial packages can be somewhat constraining and inflexible, and often fail to incorporate modern tools for design optimization and causal inference. By contrast, the writing of bespoke open-source computer code for each new design is often inefficient, work-intensive, and requires a user with significant expertise. In this session, we will hear from 3 speakers who have extensive experience in both the methodological and computational aspects of this tradeoff. Speaker 1 will provide an overview of recent advances in computational power to manage the complexity of the Bayesian adaptive approach, contrasting it with frequentist approaches. They will then describe a new commercial computing platform whose application programming interface (API) facilitates effective communication among diverse stakeholders, and presents complex Bayesian results in an intuitive manner. The platform also incorporates a novel ensemble estimation approach that leverages causal machine learning (ML) methods to enhance the detection and assessment of heterogeneity in adaptive trials. The framework uses conformal prediction to assess uncertainty in its ML estimates for finite samples, facilitating a more nuanced understanding of how different patient subgroups respond to treatments. Two new ensemble methods (a stacked X-Learner and consensus-based averaging) will be shown to produce more reliable conditional average treatment effects (CATEs). Speaker 2 will then offer a contrasting view via an in-depth exploration of the unique challenges of adaptive platform designs, and their practical implementation via a freely-available R package. This talk will emphasize the critical importance of simulating the exact platform trial one plans to conduct, and of accounting for the addition and removal of new treatments over time, as well as other sources of variation that may impact the performance of the platform. This speaker will outline the design of a hypothetical platform trial, illustrating the building and execution of a simulation plan using open-source tools. Additional topics to be covered include the concept of patient simulation, and utilization of custom patient simulators, new analysis approaches, and visualization tools using open-source tools not available in the package itself. Finally, Speaker 3 (FDA) will summarize the first two presentations, offer a regulatory perspective on the commercial and open-source approaches, give their view on the future of AI in regulatory science, and suggest areas for future work. The session's primary goals are to explore the tradeoffs between powerful commercial software packages and "home-brewed" approaches that embrace the free open-source software movement, and to assess the likely reception and possible pitfalls that sponsors may experience with each approach when used in regulatory science. The proposed session is timely (as the cost of clinical trials continues to grow and the free software movement continues to burgeon), and should appeal to a large collection of biostatisticians and clinical trialists hoping to use modern adaptive and causal inference tools on a routine basis, especially in rare and pediatric drug development.