Only Closed Testing Procedures Are Admissibility for Controlling False Discovery Proportions

• Activity Number: 368 - AOS Lecture
• Type: Invited
• Date/Time: Thursday, August 12, 2021 : 12:00 PM to 1:50 PM
• Sponsor: IMS
• Author(s): Jelle Goeman* and Aldo Solari and Jesse Hemerik
• Companies: Leiden University Medical Center and University of Milano-Bicocca and Wageningen University & Research
• Keywords: selective inference; post selection inference; multiple testing; false discovery proportion; fwer

Abstract

We consider a general class of procedures controlling the tail probability of the number or proportion of false discoveries, either in a single (random) set or in several such sets simultaneously. This class includes, among others, (generalized) familywise error, false discovery exceedance, simultaneous false discovery proportion control, and other selective inference methods. We put these procedures in a general framework, formulating all of them as special cases of true discovery guarantee procedures. We formulate both necessary and sufficient conditions for admissibility. Most importantly, we show that all such procedures are either a special case of closed testing, or they can be uniformly improved by a closed testing procedure. The practical value of our results is illustrated by giving uniform improvements of several recently proposed selective inference procedures, achieved by formulating them as a closed testing procedures. In particular, we investigate when procedures controlling conditional familywise error rate and data-splitting methods, can be uniformly improved by closed testing.