Online Program

Benchmarking Healthcare Provider Performance: Some Statistical Considerations

*Susan Paddock, RAND Corporation 

Keywords: hierarchical model, provider profiling, statistical benchmark, histogram, empirical distribution function

Setting performance standards for healthcare providers is central to efforts to improve the quality of health care. A typical approach is to analyze data to determine performance targets, or statistical benchmarks, to define high-quality care. Statistical benchmarks are typically set to reflect a high level (e.g., top 10%) of observed provider performance. Widely-use statistical benchmarking approaches are prone to over-identifying small hospitals as top performers, since they are based on raw hospital performance estimates, while benchmarking based on posterior means derived from a hierarchical model risks over-correcting for smaller providers. One compromise is to base the benchmark on the histogram of the provider-specific parameters obtained from a Bayesian hierarchical model. In a comparison of benchmarking approaches applied to Medicare Hospital Compare data, there was variation as to which hospitals exceeded the top 10% performance benchmark, but such differences were not found at the 50% threshold. Benchmarks derived from the histogram of provider performance under a hierarchical Bayesian model provide a compromise between those based on direct (raw) estimates, which are overdispersed relative to the true distribution of provider performance, and those based on posterior means, for which over-shrinkage and under-dispersion relative to the true provider performance distribution is a concern.