Avoiding Errors: Causal Effect Estimates for the Evaluation of Quality of Care Over Many Centers

• Activity Number: 40
• Type: Contributed
• Date/Time: Sunday, August 4, 2013 : 2:00 PM to 3:50 PM
• Sponsor: Health Policy Statistics Section
• Author(s): Els Goetghebeur*+ and Bart Van Rompaye and Machteld Varewyck and Stijn Vansteelandt
• Companies: Ghent University and Ghent University and Ghent University and Ghent University
• Keywords: quality of care ; causal inference ; balanced testing ; funnel plot ; bench marking ; double robust

Abstract

To evaluate quality of care based on registered patient-specific data, we contrast an expected potential outcome (risk) under treatment in specific center c, with an average expected risk across all centers. Different causal effect estimates may enter into this measure. We consider (doubly robust) (structural) regression models and different forms of penalization when necessary to reduce small center bias. Effect size and sampling variation turn into relative evidence against the null versus a meaningful alternative, and a test that optimizes a given balance between sensitivity and specificity. The approach can yield substantially different lines on the funnel plot for flagging centers with likely meaningfully higher or lower mortality. It happens with small and high volume centers where small effects may fast be statistically significant. Our test emphasizes type I error when results are made public and power when confidential feedback yields early warnings. We apply our bench marking to 1) a Belgian registry on rectal cancer designed for confidential feedback, containing small centers and 2) a Swedish stroke registry used for public feedback on high volume centers.