Closed Testing and Admissibility of Procedures Controlling False Discovery Proportions

• Activity Number: 381 - Recent Advances in Multiple Testing and False Discovery Rate Analysis
• Type: Invited
• Date/Time: Tuesday, July 30, 2019 : 2:00 PM to 3:50 PM
• Sponsor: IMS
• Author(s): Jelle Goeman* and Jesse Hemerik and Aldo Solari
• Companies: Leiden University Medical Center and University of Oslo and University of Milano-Bicocca
• Keywords: selective inference; false discoveries; multiple testing

Abstract

We consider a very general class of procedures controlling the tail probability of the number or proportion of false discoveries, either in a single (random) set or in several such sets simultanously. This class includes, among others, (generalized) familywise error, false discovery exceedance, simultaneous false discovery proportion control, and several selective inference methods. We put these procedures in a general framework, formulating all of them as giving a simultaneous lower confidence bound on the number of correctly identified discoveries in all possible subsets of the multiple testing problem. For such true discovery guarantee procedures we formulate both necessary and sufficient conditions for admissibility. First, we show that all such procedures are either a special case of closed testing, or can be uniformly improved by a closed testing procedure. Second, we show that a closed testing procedure is admissible as a true discovery guarantee procedure if and only if all its local tests are admissible. The practical value of our results is illustrated by giving a uniform (and substantial) improvement by closed testing of a recently proposed selective inference procedure.