Is it ‘random’ or ‘haphazard’? Demonstrating Effects of Nonrandom Allocation by Simulation

• Activity Number: 296 - SPEED: Biometrics - Methods and Application, Part 1
• Type: Contributed
• Date/Time: Tuesday, July 30, 2019 : 8:30 AM to 10:20 AM
• Sponsor: Biometrics Section
• Author(s): Penny Reynolds*
• Companies: University of Florida College of Medicine
• Keywords: random allocation; pre-clinical; simulation; p-values; F-distributions

Abstract

Randomization is the cornerstone of statistical inference, but frequently misunderstood and misapplied. Re-analyses of published preclinical data and simulations were used to show consequences of non-randomization for test statistic performance and underlying distributions. Fifty-nine percent of 136 published swine studies claimed random allocation of subjects; if so under the null hypothesis, baseline means should not differ and expected p-value distributions should be uniform. Summary statistics and p-values were obtained for all studies reporting baseline data for two or more groups, and aggregated using Stouffer-Liptak methods. Calculated p-values showed unexpected over-representation of small values, suggesting uncorrected trend effects. Simulations on data for 36 subjects examined the effects of true vs pseudo-randomization (alternation, false “blocking”) on error estimates and F-distributions in the presence of systematic trend. True randomization protected against systematic trend, but pseudo-randomization resulted in reference distribution ‘collapse’ to a single value, invalidating inference tests. Inferences based on non-randomized data will be erroneous.