|
Abstract:
|
Effective scheme evaluation is paramount in ensuring road safety interventions are operating effectively and ensuring future policy decision making is best informed. The most common methods for this (Empirical Bayes and Full Bayes) make use of a control group of sites to inform the Bayesian model for the treatment effect at each treated site. The issue of how best to select the control group is rarely acknowledged in literature or practice. Here we demonstrate this effect using simulated data, observing a clear and substantial increase in treatment effect bias with increasing dis-similarity between treatment and control groups. We demonstrate an improved method for incorporating control data, propensity score weighted regression, whereby the contribution of each control site in the model for each treated site is weighted by their similarity. We determine similarity by calculating propensity scores for each site regressing on whether the site was treated, and using propensity score difference as a metric for similarity. This approach allows bespoke models for each treated site to be formed, with weight given only to the control sites which are most exchangeable to the treated site.
|
