.png)
168 – Statistical Literacy 2013
Randomization Metrics: Jointly Assessing Predictability and Efficiency Loss in Covariate Adaptive Randomization Designs
Dennis Sweitzer
Medidata Solutions
Randomization methods generally are designed to be both unpredictable and balanced between treatment allocations overall and within strata. However, when planning studies, little consideration is given to measuring these characteristics, nor are they examined jointly, and published comparisons between methods often are not useful.
In order to compare randomization performance, we simulated various covariate-adjusted randomization methods (i.e., permuted block, and dynamic allocation), and compared measures of balance and randomness both graphically and statistically.
To measure predictability, we modified the Blackwell-Hodges potential selection bias in which an observer guesses the next treatment to be one that previously occurred least in a strata, reflecting a game theory model pitting observers versus statistician, and is easy to calculate and interpret.
To measure imbalances, we calculated efficiency loss using Atkinson's method because the main impact of imbalances is a loss of statistical power, and even if treatments are balanced overall, imbalances within small strata can have a disproportionate impact on efficiency, and is interpretable as lost sample size.