Benchmarking Healthcare Provider Performance: Some Statistical Considerations
*Susan Paddock, RAND Corporation
Keywords: hierarchical model, provider profiling, statistical benchmark, histogram, empirical distribution function
Setting performance standards for healthcare providers is central to efforts to improve the quality of health care. A typical approach is to analyze data to determine performance targets, or statistical benchmarks, to define high-quality care. Statistical benchmarks are typically set to reflect a high level (e.g., top 10%) of observed provider performance. Widely-use statistical benchmarking approaches are prone to over-identifying small hospitals as top performers, since they are based on raw hospital performance estimates, while benchmarking based on posterior means derived from a hierarchical model risks over-correcting for smaller providers. One compromise is to base the benchmark on the histogram of the provider-specific parameters obtained from a Bayesian hierarchical model. In a comparison of benchmarking approaches applied to Medicare Hospital Compare data, there was variation as to which hospitals exceeded the top 10% performance benchmark, but such differences were not found at the 50% threshold. Benchmarks derived from the histogram of provider performance under a hierarchical Bayesian model provide a compromise between those based on direct (raw) estimates, which are overdispersed relative to the true distribution of provider performance, and those based on posterior means, for which over-shrinkage and under-dispersion relative to the true provider performance distribution is a concern.
Important Dates & Deadlines
- October 9 - 11, 2013