Online Program

Assessing Cluster Variation through Profiling Using Reference Effect Measures (307863)

*Thomas Jacob Glorioso, Department of Veterans Affairs 
Gary K Grunwald, University of Colorado, Denver 
P Michael Ho, Veterans Affairs 
Wenhui Liu, Veterans Affairs 

Keywords: Cluster variation, reference effect measures, hierarchical models, multi-level data, generalized linear mixed models, random effects

Profiling is commonly used with multi-level data to identify low and high performing clusters, e.g. hospitals, providers, etc. A related question that can be addressed by profiling involves quantifying cluster variation, especially in models with non-normal outcomes. As with other methods using random effects, cluster estimates in profiling are subject to shrinkage, especially among small clusters with large deviations from overall averages. Thus, the cluster estimates provided by profiling may lead us to understate the true amount of cluster variation. To address this issue, we propose the application of a Reference Effect Measure approach. A set of hypothetical clusters at specified percentiles of the random effect distribution is profiled, giving a smooth estimate of the cluster distribution on the profiling scale. Using this approach, cluster variation is quantified on an easily interpretable scale, such as probabilities or risk ratios for binary outcomes, and is less susceptible to shrinkage and underestimation. We apply this approach to assess variation in cardiology care across hospitals in the VA Clinical Assessment Reporting and Tracking program.