Casting the Bones: The Worth and Potential Worthlessness of Subgroup Analyses

• Activity Number: 164
• Type: Topic Contributed
• Date/Time: Monday, August 10, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Kevin Buhr*
• Companies: University of Wisconsin - Madison
• Keywords: clinical trials ; subgroup analysis

Abstract

At JSM 2014, Stephen Stigler shared "Seven Pillars of Statistical Wisdom", seven profound ideas forming the foundation of our discipline. When we design and analyze a clinical trial to answer a clinical question, we bring the pillars of Design and Aggregation to bear in the service of medical science. When we deaggregate the data to perform subgroup analyses not anticipated by the design, we step away from statistical principles in favor of ad hoc approaches. Remarkably, even as statisticians, we overestimate the worth of such analyses. We understand the theoretical risk of spurious results, but in practice consider spuriousness a "possible explanation" rather than "the most likely explanation". We justly refuse to use post hoc results for hypothesis testing but remain too optimistic when using them for hypothesis generation. We know that analysis of all available data is most definitive, but we consider consistencies across subgroups comforting and inconsistencies compelling. In this talk, we'll review some of the pitfalls and discuss ways of conceptualizing and presenting these analyses (at least to ourselves) to help put them into the appropriate context.