Balance Reduction for Observational Studies Using Propensity Scores
*Thomas Ezra Love, Case Western Reserve University 

Keywords: propensity scores, matching, weighting, observational studies, Swan-Ganz catheter

The design of an observational study can mimic the approach of a randomized clinical trial in many ways, barring, of course, the most important element - randomization. Propensity scores and related methods (like the prognostic score) have been used to help alleviate observable selection bias in comparative effectiveness studies for some time. We present a new approach to assessing the impact of propensity score analyses on selection bias reduction in observational studies which combines the use of dot plots with more sophisticated assessments of distributional similarity. So-called Love plots use familiar graphical forms to help identify potential problems with pre- and post-adjustment standardized differences (or similar metrics) in the means of observational study covariates where matching, weighting or other approaches are planned to account for observable confounding by indication.

A potential weakness of this approach is that the choice of metric inherently focuses attention on specific summaries of the distribution of the covariates involved that may or may not be sufficient for the eventual analytic task. We propose a new plot, making use of several augmentations which may help to alleviate the problem that balance in standardized differences may be insufficient to declare the covariates sufficiently balanced to merit further regression-based analyses without heroic assumptions. We demonstrate the use of the resulting plots in the design of several analytic approaches for assessing the impact of a particular medical device (the Swan-Ganz catheter) on a variety of outcomes in the context of the SUPPORT study, for which fairly complete data are available to the public.