Massive Multiple Testing Does Not Increase Rates of Spurious Findings in Genetic Association Studies

• Activity Number: 514
• Type: Contributed
• Date/Time: Wednesday, August 6, 2014 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistics in Epidemiology
• Author(s): Dmitri Zaykin*+ and Olga Vsevolozhskaya and Chia-Ling Kuo and Luda Diatchenko
• Companies: National Institute of Environmental Health Sciences and Michigan State University and University of Connecticut and McGill University
• Keywords: genetic association ; false discovery rate ; multiple testing

Abstract

Imagine an utterly uninformed epidemiologist who is studying effects of various predictors on susceptibility to a disease. The epidemiologist is oblivious to any external knowledge regarding possible effects of predictors on the outcome and simply tests every predictor in sight. In this scenario, predictors that do in fact influence the outcome, i.e., "true signals" occur at a constant rate. In other words, the rate of true signal occurrence does not diminish as additional predictors are tested. At the end of the day, the epidemiologist reports the predictor yielding the smallest P-value as a potentially true signal. This strategy is often perceived with disdain as "data torturing". However, a predictor with the smallest P-value in such a study becomes increasingly less likely to be a spurious association as more tests are performed. More generally, a set of "top hits" in a multiple-testing experiment becomes increasingly enriched with true signals as more tests are carried out. To understand and quantify this phenomenon, we develop statistical theory and practical ways to estimate rates of spurious findings expected in a set of predictors with the smallest P-values. Our results ca