Best Practices for Detecting Treatment Effect Heterogeneity in Multisite Trials

• Activity Number: 65 - New Methods for Identifying and Testing Heterogeneous Treatment Effects in One or a Pair of Studies
• Type: Topic Contributed
• Date/Time: Sunday, July 28, 2019 : 4:00 PM to 5:50 PM
• Sponsor: Health Policy Statistics Section
• Author(s): Luke Miratrix*
• Companies: Harvard University
• Keywords: cross-site variation; multisite trials; treatment heterogeniety; treatment variation; multilevel modeling; meta analysis

Abstract

Treatment effect heterogeneity is a critical component in understanding the results of large-scale randomized trials. A first step in an analysis of such variation might be to test for the presence of variation overall (i.e., to test for idiosyncratic variation not fully modeled by covariates) before tying the variation to specific covariates (to ideally obtain a model of systematic, or explainable, variation). The question is then how to conduct such an initial first-step omnibus test in a maximally powerful way. This talk first compares two classic methods for detecting such variation without covariates, and then extends these methods to take advantage of site level covariates that might partially predict such variation. We then propose a hybrid test that tests for both systematic and idiosyncratic variation simultaneously using an adjusted likelihood ratio test. Overall, we examine two primary methodological research questions: (1) What methods are most powerful for detecting cross site variation, and why? and (2) Can one exploit a covariate modestly predictive of variation to improve the power of an overall test?