Optimal and Maximin Procedures for Multiple Testing Problems

• Activity Number: 471 - Advances in High-Dimensional Inference and Multiple Testing
• Type: Contributed
• Date/Time: Wednesday, July 31, 2019 : 8:30 AM to 10:20 AM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Saharon Rosset* and Ruth Heller and Amichai Painsky and Ehud Aharoni
• Companies: Tel Aviv University and Tel-Aviv University and Hebrew University Jerusalem and IBM Research
• Keywords: Multiple testing; Infinite dimensional optimization; Neyman Pearson

Abstract

Multiple testing problems are a staple of modern statistics. The fundamental objective is to reject as many false null hypotheses as possible, subject to controlling an overall measure of false discovery, like family-wise error rate (FWER) or false discovery rate (FDR). We formulate multiple testing of simple hypotheses as an infinite-dimensional optimization problem, seeking the most powerful rejection policy which guarantees strong control of the selected measure. We show that for exchangeable hypotheses, for FWER or FDR and relevant notions of power, these problems lead to infinite programs that can provably be solved. We explore maximin rules for complex alternatives, and show they can be found in practice, leading to improved practical procedures compared to existing alternatives. We derive explicit optimal tests for FWER or FDR control for three independent normal means. We find that the power gain over natural competitors is substantial in all settings examined. We apply our optimal maximin rule to subgroup analyses in systematic reviews from the Cochrane library, leading to an increased number of findings compared to existing alternatives. https://arxiv.org/pdf/1804.10256