Methodological Challenges of Measuring Physician Performance

• Activity Number: 409
• Type: Contributed
• Date/Time: Wednesday, August 6, 2003 : 2:00 PM to 3:50 PM
• Sponsor: Section on Health Policy Statistics
• Author(s): James E. Bost*+
• Companies: University of Arkansas for Medical Sciences
• Keywords: measurement ; sample size ; risk adjustment ; psychometrics ; physicians

Abstract

There are many statistical challenges associated with measuring the quality of care provided by physicians. First, formulas for determining appropriate sample sizes (no. of patient data per physician) depend on whether results will be used to compare physicians to each other (report cards), to a benchmark (pay for performance) or for recognition (meeting a minimum criteria). Second, whether or not to case mix adjust depends on whether process or outcome variables are being used. Case mix or risk adjustment depends also on the availability of patient comorbidities, the balance between sample size and number of variables, and using the right model (when are hierarchical models appropriate). Third, with small sample sizes, choosing the best method, if any, to impute missing data is crucial. Using propensity scores or item response theory techniques haS recently been explored. Finally, many believe the development of composite measures is the best path to success. Using composites can increase precision and reduce sample size. New approaches to analyzing composites include hierarchical and longitudinal modeling, latent variable modeling, and multivariate signal extraction.