Assessing Replicability Across Studies: The R-Value

• Activity Number: 471
• Type: Invited
• Date/Time: Wednesday, August 6, 2014 : 10:30 AM to 12:20 PM
• Sponsor: ASA
• Author(s): Ruth Heller*+ and Yoav Benjamini and Marina Bogomolov
• Companies: Tel Aviv University and Tel Aviv University and Technion - Israel Institute of Technology
• Keywords: Replicability Analysis ; Meta Analysis ; Multiple Comparisons ; Adjusted p-value ; False Discovery Rate ; Genome-wide association studies

Abstract

Studies that examine each multiple features are combined for two purposes: (1) to increase the number of cases in order to increase the power to detect a feature; (2) to establish replicability, thereby addressing the basic dogma of science that a finding is more convincingly a true finding if it is replicated in at least one more study. Existing meta-analysis methods address only purpose (1). To address purpose (2), we propose formal statistical methods to declare that findings have been replicated across studies. To quantify the strength of replication, we compute for each feature the FDR r-value, i.e. the lowest FDR level at which we can say that the finding is among the replicated ones, or the FWER r-value, which is similarly defined. We show the theoretical guarantees for both FDR and FWER control on replicability claims for independent and dependent data within each study. We demonstrate the usefulness of our approach for establishing replicability of associations in ``omics" research.