299 – Meta-Analysis and Other Innovative Comparison
Tree-Structured Reliability Analysis for Magnetic Resonance Imaging (MRI) Data
Ruji Yao
Hanzhe Zheng
Merck Research Laboratories
Inter-rater reliability refers to a comparison of scores assigned to the same target by two or more raters. The simplicity of the two-way fixed effect model has rendered it a popular method for reliability estimation. On the other hand, "trees" - a class of non-parametric methods used to break partition-able data into several pieces (nodes), allows each node to then be fit with most suitable method. Here, we present an intuitive tree-based approach for reliability estimation. In a recent clinical trial, we used MRI data to evaluate the treatment effect on subjects with active axial spondyloarthritis. Each MRI slide was scored by two independent raters and the average of the two scores was used as an endpoint. In a routine reliability check, we noticed several intuitively incorrect reliability results as compared with the raw data. Motivated by several tree-based methods, we partition the response data to create a simple 2-node tree; we then combine the results with a modified reliability formula. With this new formula, the reported reliability scores follow intuitively from the raw data and also provide additional insight into the source of the variance of the MRI data itself.
