Keywords: selective inference, clinical safety, false discovery rate, generalized familywise error rate, multiple testing, multiple families
Many heath related research, such as clinical safety studies, often involve testing multiple families of hypotheses. Most existing multiple testing methods cannot guarantee strong control of appropriate type 1 error rates suitable for such increasingly complex research questions. We will introduce a novel two-stage procedure based on the recently developed idea of selective inference for clinical safety studies. In the first stage, some significant families are selected by using some family-level global test, which guarantees control of generalized familywise error rate among the selected families. In the second stage, individual hypotheses are tested for each selected families, which guarantees control of conditional false discovery rate based on the fact that the family is selected. By applying the proposed procedure to clinical safety studies, one can not only efficiently flag the significant clinical adverse events but also select body systems of interest as extra information for further research. Our simulation studies show that the proposed procedure can be more reliable than alternative methods such as Double FDR in the setting of clinical safety.