Abstract:
|
Student, aka William Gossett, noted empirically that when the null hypothesis was true, that errors in estimating the difference in means resulted in a reciprocal error in estimating the standard error of the difference in means. This observation was published posthumously and overlooked by almost everyone for a century. While this leaves the usual t-test formula and analysis of variance appropriate for testing the alternative hypothesis, it is not for evaluating the null hypothesis. Under some circumstances this can make a large difference in the resulting p-values and decision about whether the results are "statistically significant". The problem is clear with appropriate formulas for t-values and for the distribution of the total corrected sums of squares when the data comes from two samples. In a uniform distribution of 10 values divided into two equal sized samples, it can result in as much as a 30-fold error in p-values. This omission could be playing some role in the poor repeatability of small experiments and meta analyses. Inappropriate blurring of the difference between evaluation of the null and alternative hypotheses maybe either a cause or effect of this failure.
|