Alternatives to the Pocock-Simon method for trickle-in random assignment
*Ryan T. Moore, University of California, Berkeley 

Keywords:

Blocking uses observed covariates to sort experimental units into homogeneous sets prior to randomization to treatment conditions. Pocock and Simon (1975) describe a general approach to blocking in experiments with sequentially assigned treatment conditions (also called ``trickle-in assignment'). Here, we implement several alternatives to their example approaches; our alternatives have been tailored to a pilot study of the effects of a manipulation on patients with post-traumatic stress disorder (PTSD).

We first use simulation studies to demonstrate that sequentially-blocked experiments result in better covariate balance and greater precision in the estimate of the treatment effect. We then compare several sequential blocking algorithms and show how we selected a particular algorithm for implementation in the PTSD experiment. We show the balance produced by this algorithm in the pilot study, and compare it to the expected balance from complete randomization. Finally, because subjects are often processed for randomization by varyingly-trained research assistants in many sequential experiments, we offer a general query-based interface in R to minimize intake error.

In particular, we consider the average Mahalanobis distance between a to-be-assigned unit and the already-assigned units in a given treatment condition. We use this measure in several ways to bias the new treatment assignment toward conditions that look different from the new unit. We employ the chi-squared statistic of the omnibus d^2 measure to show improvements in balance (Hansen and Bowers 2008) and the variance from the linear contrast of Atkinson (2003) to show improvements in causal estimate precision. No significant discernible differences arise when we use few versus several covariates, highly correlated versus uncorrelated covariates, or a multivariate normal versus bimodal sample of units. We also consider performance when covariate outliers show up early, late, or in the middle of an experiment. We show that sequential blocking most outperforms complete randomization when the treatment effect is moderately difficult to detect.