Sequential Tests of Multiple Hypotheses Controlling False Discovery Rate

• Activity Number: 361
• Type: Contributed
• Date/Time: Tuesday, August 6, 2013 : 10:30 AM to 12:20 PM
• Sponsor: IMS
• Author(s): Jay Bartroff*+ and Jinlin Song
• Companies: USC and USC
• Keywords: false discovery rate ; sequential analysis ; multiple comparisons ; multiple testing ; Wald approximations

Abstract

Because much real world scientific experimentation (a) unfolds sequentially in time, and (b) involves multiple comparisons, until recently there has been an unmet need for flexible sequential methods that control multiple testing error rates. In this talk we give a general framework for combining a battery of arbitrary sequential hypothesis tests into a sequential multiple testing procedure that controls the false discovery rate (FDR), and another that simultaneously controls the FDR and its type II analog, the false non-discovery rate. In both versions, dramatic savings in expected sample size can be achieved relative to fixed-sample procedures since the sequential versions can "drop" individual data streams as soon as additional sampling is no longer needed to reach a decision regarding its corresponding hypothesis. The sequential procedure enjoys many of the same attractive properties of the seminal Benjamini-Hochberg (1995) procedure, including provable FDR control under arbitrary dependence of the data streams. To ease implementation we give closed-form expressions for critical values to use with some common sequential tests, and discuss applications.