How Do Widely Used Statistical Benchmarking Approaches Compare to Optimal Estimation Under Hierarchical Bayesian Modeling?

• Activity Number: 143
• Type: Invited
• Date/Time: Monday, July 30, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Health Policy Statistics Section
• Author(s): Susan Paddock*+
• Companies: RAND Corporation
• Keywords: statistical benchmark ; profiling ; Bayesian hierarchical model ; histogram ; empirical distribution function ; health care

Abstract

Statistical benchmarks are central to health care provider quality improvement efforts. They are typically defined as the 90th percentile of the raw provider performance scores or similar, and tend to over-identify small, low-volume hospitals as top performers. One previously proposed alternative is to estimate the benchmark using posterior means derived from a hierarchical model. However, this may over-compensate for smaller, higher-variance providers. A compromise is provided by the histogram of the provider-specific parameters from a Bayesian hierarchical model. The posterior mean histogram is optimal under integrated squared error loss for estimating the distribution of provider-specific performance. In this talk, I use Medicare Hospital Compare data to examine the performance of histogram-based statistical benchmarks relative to more widely-used methods, showing whether and how performance differs across methods. The stability of conclusions drawn about provider performance across methods and policy implications for health care quality improvement efforts such as pay-for-performance are discussed.