On the Bayesian Credible Intervals for Intraclass Correlation Coefficients with Small Number of Raters

• Activity Number: 398
• Type: Topic Contributed
• Date/Time: Tuesday, August 2, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Dai Feng*+ and Valdimir Svetnik and Alexandre Coimbra and Richard Baumgartner
• Companies: Merck & Co., Inc. and Merck & Co., Inc. and Merck & Co., Inc. and Merck & Co., Inc.
• Keywords: Intralclass correlation coefficient ; small number of raters ; Bayesian credible interval

Abstract

In drug development, an important task is to study reproducibility or test-retest reliability. Intraclass correlation coefficient (ICC) is a metric that has been widely used to assess the reproducibility. In particular, the ICC obtained from a mixed effects model with fixed rater effects is recommended in the situation with a small number of raters, which is a typical setup in early drug development studies. To calculate confidence intervals (CIs) for the ICC, various frequentist methods have been proposed. They include methods based on second and higher moment approximations, the delta method, and others. We propose using a Bayesian method with a Jeffreys' prior to obtain the credible sets. When there are two raters, the independent samples can be generated from constructive posteriors and obtained very quickly using vectorized computation in R. Judging by simulation studies and results on real EEG datasets, the Bayesian approach is at least comparable with and sometimes better than frequentist approaches based on different frequentist properties. The Bayesian method should be considered as an alternative for ICC CI calculation in early drug development.