Online Program Home
My Program

Abstract Details

Activity Number: 145
Type: Invited
Date/Time: Monday, August 1, 2016 : 10:30 AM to 12:20 PM
Sponsor: Health Policy Statistics Section
Abstract #318153 View Presentation
Title: Hierarchical Regression and Variance-Function Modeling to Estimate the Inter-Rater Intraclass Correlation Coefficient in Assessments of Shared Decision Making
Author(s): James O'Malley* and Paul J. Barr and Glyn Elwyn
Companies: Geisel School of Medicine at Dartmouth and Geisel School of Medicine at Dartmouth and Geisel School of Medicine at Dartmouth
Keywords: Bayesian ; Hierarchical model ; Intraclass correlation coefficient ; Option5 ; Shared decision making ; Variance function

We describe a novel methodological approach to assessing the inter-rater reliability of assessments of the amount of shared decision-making in a patient-physician clinical encounter. Two-raters each assess recordings of patient-physician encounters across three clinical sites using the Option5 shared-decision-making tool. The desired output is the interrater intra-class correlation coefficient (ICC) of the OPTION5 scores, accounting for heterogeneity between studies and heteroscedasticity of ratings with respect to the true amount of shared decision-making. We describe a three-level hierarchical model with random effects for patient-physician encounter and clinical site, covariates for rater and other predictors, and a variance-mean function and a Bayesian estimation method. We show that the ICC varies widely depending on whether the encounters being distinguished are restricted to the same site and to the amount of shared decision making in the encounter. Because current applied practice in shared decision often ignores these subtleties, we argue for the introduction of standards regarding the definition of ICC and other measures of inter-rater reliability in this field.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2016 program

Copyright © American Statistical Association