Detecting Multiple Replicating Signals Using Adaptive Filtering Procedures

• Activity Number: 165 - SLDS CSpeed 2
• Type: Contributed
• Date/Time: Tuesday, August 10, 2021 : 10:00 AM to 11:50 AM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Lin Gui* and Jingshu Wang and Weijie Su and Chiara Sabatti and Art Owen
• Companies: The University of Chicago and The University of Chicago and University of Pennsylvania and Stanford University and Stanford University
• Keywords: simultaneous signals; high-throughput experiments; partial conjunction; composite null; multiple testing

Abstract

Replicability is a fundamental quality of scientific discoveries: we are interested in those signals that are detectable in different laboratories, study populations, across time, etc. Unlike meta-analysis which accounts for experimental variability but does not guarantee replicability, testing a partial conjunction (PC) null aims specifically to identify the signals that are discovered in multiple studies. In many contemporary applications, e.g., comparing multiple high-throughput genetic experiments, a large number M of PC nulls need to be tested simultaneously, calling for a multiple comparisons correction. However, standard multiple testing adjustments on the M PC p-values can be severely conservative, especially when M is large and the signals are sparse. We introduce AdaFilter, a new multiple testing procedure that increases power by adaptively filtering out unlikely candidates of PC nulls. We prove that AdaFilter can control FWER and FDR as long as data across studies are independent, and has much higher power than other existing methods.