East Coast Ballroom
A cautionary note about assessing heterogeneity over outcome scores in randomized trials (307884)
Issa Dahabreh, Brown University*Hongseok Kim, Brown University
Keywords: Heterogeneity of treatment effects, effect modification, outcome score, effect score, risk score
Assessing heterogeneity of treatment effects (HTE) is important for tailoring treatment plans and policy interventions. Some investigators have strongly advocated for the use of “outcome score” (or “risk score”) procedures for the assessment of HTE over multiple patients’ characteristics. These procedures involve (1) developing a predictive model of the outcome in the control group or the entire trial sample, given baseline covariates; (2) stratifying the trial sample by cut points (e.g., quantiles) of predicted values from the model; and (3) assessing HTE over the strata using various statistical tests. In this study, we show that, under certain data laws, outcome score procedures can lead to the misleading conclusion that heterogeneity is absent, even in very large randomized trials with very heterogeneous treatment effects. Furthermore, we show that conceptually simple and easy to implement methods that assess HTE over “effect scores” (i.e., predicted treatment effects) do not have the same drawbacks. We conclude that “outcome score” procedures for the assessment of HTE should not be the primary approach to assessing HTE when analyzing data from randomized trials.