Preventing False Discovery of Heterogeneous Treatment Effect Subgroups in Randomized Trials

• Activity Number: 364 - Contributed Poster Presentations: Section on Medical Devices and Diagnostics
• Type: Contributed
• Date/Time: Tuesday, July 31, 2018 : 10:30 AM to 12:20 PM
• Sponsor: Section on Medical Devices and Diagnostics
• Author(s): Joseph Rigdon* and Michael Baiocchi and Sanjay Basu
• Companies: Stanford University and Stanford University and Stanford University School of Medicine
• Keywords: classification and regression trees; decision support tool; heterogeneous treatment effects; matching

Abstract

Current methods suffer from a potential for false detection of heterogeneous treatment effects (HTEs) due to imbalances in covariates between candidate subgroups. We introduce matching plus classification trees (mCART) that yields balance in covariates in identified HTE subgroups. We compared mCART to a classical method (logistic regression [LR] and two machine-learning approaches, (random forest [RF] and gradient random forest [gradient RF]), in simulations with a binary outcome with known HTE subgroups. We considered an N=200 phase II oncology trial where there were either no HTEs (1A) or two HTE subgroups (1B) and an N=6000 phase III CVD trial where there were either no HTEs (2A) or four HTE subgroups (2B). Additionally we considered an N=6000 phase III CVD trial where there was no average treatment effect (ATE) but there were four HTE subgroups (2C). All HTE subgroups identified by mCART had treated-control covariate balance with absolute standardized differences less than 0.2, whereas ASDs for other methods typically exceeded 0.2. The imbalance in covariates in identified subgroups for LR, RF and gradient RF indicates false HTE detection may have been due to confounding.