Keywords: selective inference, false discovery rate, generalized familywise error rate, multiple testing, hierarchical testing, clinical safety
Many complex biomedical studies, such as clinical safety studies and genome-wide association studies, often involve testing hierarchically ordered families of hypotheses. Most existing multiple testing methods cannot guarantee strong control of appropriate type 1 error rates suitable for such increasingly complex research questions. In this talk, we will introduce a novel two-stage procedure based on the recently developed idea of selective inference for clinical safety studies. In the first stage, some significant families are selected by using some family-level global test, which guarantees control of generalized familywise error rate (k-FWER) among the selected families. In the second stage, individual hypotheses are tested for each selected families, which guarantees control of conditional false discovery rate (FDR) based on the fact that the family is selected. By applying the proposed procedure to clinical safety studies, one can not only efficiently flag the significant clinical adverse events (AEs) but also select body systems of interest (BSoI) as extra information for further research. Our simulation studies show that the proposed procedure can be more reliable than alternative methods such as Double FDR in the setting of clinical safety.