Randomization is the cornerstone of statistical inference, but frequently misunderstood and misapplied. Re-analyses of published preclinical data and simulations were used to show consequences of non-randomization for test statistic performance and underlying distributions. Fifty-nine percent of 136 published swine studies claimed random allocation of subjects; if so under the null hypothesis, baseline means should not differ and expected p-value distributions should be uniform. Summary statistics and p-values were obtained for all studies reporting baseline data for two or more groups, and aggregated using Stouffer-Liptak methods. Calculated p-values showed unexpected over-representation of small values, suggesting uncorrected trend effects. Simulations on data for 36 subjects examined the effects of true vs pseudo-randomization (alternation, false “blocking”) on error estimates and F-distributions in the presence of systematic trend. True randomization protected against systematic trend, but pseudo-randomization resulted in reference distribution ‘collapse’ to a single value, invalidating inference tests. Inferences based on non-randomized data will be erroneous.