All Times ET
Keywords: applied statistics, ANOVA, Regression, non-normal, transformation
ANOVA and Regression are classic mainstays of statistical analysis, and they are sometimes perceived as "robust," especially with respect to non-normal residuals. However, non-normality may lead to an opportunity cost whereby these common parametric models produce lower power than alternative models do. Unfortunately, the relative performance of alternatives has been unclear. In 4 Monte Carlo studies, we compared the power and Type I error rates of several alternatives, including non-parametric rank-based regression, aligned ranks, MM estimation with or without a fast robust bootstrap, and a variety of data transformation methods. Methods were compared across a variety of study designs and distribution skewness and kurtosis values. The most powerful methods involved an inverse normal transformation (INT), which leads to approximate normality regardless of the initial distribution shape. INT has been variously referred to as rankit, van der Waerden score, Blom score, normal score, or rank-based inverse normal (RIN) transformation. In most scenarios, power was highest when ANOVA or regression was performed on a Direct INT of y, an Indirect INT of residuals from a restricted model, or an Omnibus INT (McCaw et al., 2020) that combined Direct and Indirect INT test results. In most scenarios, the INT methods beat or tied other methods in terms of power, and INTs simultaneously maintained the intended Type I error rates. However, for interaction effects, only the indirect INT consistently avoided Type I error inflation. Additionally, results showed surprising limitations of some non-transformation methods, including low power amidst uniform residuals and Type I error inflation, sometimes even for large samples.