FDR Doesn't Tell the Whole Story: Joint Influence of Effect Size and Covariance Structure on the Distribution of the False Discovery Proportion

• Activity Number: 359
• Type: Contributed
• Date/Time: Tuesday, August 2, 2011 : 10:30 AM to 12:20 PM
• Sponsor: Biometrics Section
• Author(s): Alan H. Feiveson*+ and James Fiedler and Robert Ploutz-Snyder
• Companies: NASA Johnson Space Center and Universities Space Research Association and Universities Space Research Association
• Keywords: false discovery ; FDR ; multiple comparisons ; multiple testing ; correlation structure

Abstract

As part of a 2009 Annals of Statistics paper, Gavrilov, Benjamini, and Sarkar report results of simulations that estimated the false discovery rate (the FDR) for equally correlated test statistics using a well-known multiple-test procedure. In our study we estimate the distribution of the false discovery proportion (FDP) for the same procedure under a variety of correlation structures among multiple dependent variables in a MANOVA context. Specifically, we study the mean (FDR), skewness, kurtosis, and percentiles of the FDP distribution in the case of multiple comparisons that give rise to correlated non-central t-statistics when results at several time periods are being compared to baseline. Even if the FDR achieves its nominal value, other aspects of the distribution of the FDP depend on the interaction between signed effect sizes and correlations among variables, proportion of true nu