Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 57 - Nonparametric Modeling I
Type: Contributed
Date/Time: Sunday, August 8, 2021 : 3:30 PM to 5:20 PM
Sponsor: Section on Nonparametric Statistics
Abstract #318153
Title: Another Look at the Lady Tasting Tea and Differences Between Permutation Tests and Randomization Tests
Author(s): Jesse Hemerik* and Jelle Goeman
Companies: Wageningen University & Research and Leiden University Medical Center
Keywords: Permutation test; Lady Tasting Tea; Randomization test; Group invariance test; Small sample size

The statistical literature is very inconsistent in the use of the terms "permutation test" and "randomization test". Several authors successfully argue that these terms should be used to refer to two distinct classes of tests and that there are major conceptual differences between these classes. The present paper explains an important difference in mathematical reasoning between these classes: a permutation test fundamentally requires that the set of permutations has a group structure, in the algebraic sense; the reasoning behind a randomization test is not based on such a group structure and it is possible to use an experimental design that does not correspond to a group. In particular, we can use a randomization scheme where the number of possible treatment patterns is larger than in standard experimental designs. This leads to exact p-values of improved resolution. We discuss applications in randomized trials and elsewhere. Further, we explain that Fisher's famous Lady Tasting Tea experiment, which is commonly referred to as the first permutation test, is in fact a randomization test. This distinction is important to avoid confusion and invalid tests.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2021 program