Online Program

Trial Design Leveraging Prognostic Score Adjustment Reduces Sample Sizes While Preserving Power and Type I Error Control (302408)

*Alejandro Schuler, Unlearn.AI 

Keywords: Biostatistics, Covariate Adjustment, Disease Progression Model, AI, Machine Learning

Trials enroll a large number of subjects in order to attain power, making them lengthy and expensive. It is possible to use data from registries, prior trials, or health records to reduce sample size requirements. However, existing methods for “borrowing” historical data maintain power by sacrificing type-I error rate control. Here, we demonstrate a use of historical data that shrinks trial size while maintaining power and type I error control. Our approach is to estimate the treatment effect in the trial while adjusting for the trial subjects' predicted outcomes according to a prognostic model learned from historical data. We call this process PROCOVA (Prognostic Covariate Adjustment). Effects estimated via PROCOVA have less variance than unadjusted estimates by a factor related to the correlation coefficient between the prognostic score and the true outcome. This fact can be exploited to prospectively design trials with smaller sample sizes than a traditional sample size calculation would suggest. The sample size calculation is parsimonious because the adjustment is distilled into a single covariate. We tested PROCOVA in the context of an Alzheimer’s disease drug trial. We first built a prognostic model for our outcome of interest using two databases of historical Alzheimer’s patients. We then assessed the out-of-sample correlation coefficient between the model and outcome. This suggested that we would need a sample size of 321 to attain 80% power in a hypothetical trial with PROCOVA and 402 subjects without. We then reanalyzed data from a previously run drug trial - first by taking 402 subjects and estimating the unadjusted effect, and then by sampling 321 subjects and applying PROCOVA. The two results were qualitatively identical with no loss in certainty despite the smaller sample size in the latter case. We confirmed with simulations that trials with PROCOVA meet their design power with smaller samples than would otherwise be necessary.