519 – Contributed Oral Poster Presentations: Social Statistics Section
Covariate Balance in Propensity Score Models: Much Ado About Nothing?
Jessica Montgomery
University of South Florida
Eun Sook Kim
University of South Florida
Jeffrey D. Kromrey
University of South Florida
Rheta E. Lanehart
University of South Florida
Patricia RodrÃguez de Gil
University of South Florida
Derrick Saddler
University of South Florida
Yan Wang
University of California at Los Angeles
Conventional wisdom states that sample covariate balance is needed to obtain unbiased treatment effect estimates in propensity score models; yet the literature offers little empirical evidence of a relationship between sample covariate balance and the quality of the estimated treatment effect. The present study used simulation to investigate this relationship. The factors investigated include correlation among covariates, the strength of relationships between covariates and both treatment assignment and outcome, the number and reliability of covariates, the magnitude of the population treatment effect, sample size, and accuracy of model specification. Each sample was analyzed for both the degree of covariate balance and estimation error in the treatment effect estimate. Results indicate increased balance only yields improved estimates in naïve models. No relationship is evident between sample covariate balance and estimation error in models that adjust for covariate differences. Results are interpreted in terms of the discrepancy between sample estimates and population parameters, and the potential for sample balance estimates to provide useful information about the quality of propensity score models.