Comprehensive Evalutation of Statistical Tests for Two-Location Comparison (303223)

Keywords: t-test, Wilcoxon rank-sum test, resampling test, multi-stage test, normality test, Fleishman's power method, Tukey's ladder of transformations, contaminated data

Two-location comparison is usually carried out by t-test or Wilcoxon rank-sum test depending on if the normality assumption is met. A normality test can be applied to assess the normality and F-test for comparing the two population variances can be used to choose Welch's or pooled t-test. Previous research has compared t-test, Wilcoxon rank-sum test, and some of their multi-stage tests while using simplified simulations and ignoring other tests. We compared a selection of tests by simulation. Our work is more comprehensive than any previous research in that 1) we include almost every statistical test for two-location comparison; 2) we conduct simulation experiments assuming sample sizes for a broad range of power given a small, medium, and large effect size; 3) we simulate normal and various non-normal data following Fleishman's power method, Tukey's ladder of transformations, and the method by Marrero (1985) for contaminated data. We are particularly interested and our simulation experiments allow us to compare the tests when the sample size is small, when the population variances are unequal, and when the data can be transformed to normal data via a transformation.