Online Program Home
My Program

Abstract Details

Activity Number: 227 - Recent Advances in the Design and Analysis of Multi-Reader Imaging Studies
Type: Topic Contributed
Date/Time: Monday, July 29, 2019 : 2:00 PM to 3:50 PM
Sponsor: Section on Medical Devices and Diagnostics
Abstract #304783 Presentation
Title: Assigning Readers to Cases in Multi-Reader Multi-Case Imaging Studies Using Balanced Incomplete Block Designs
Author(s): Erich Huang*
Companies: National Cancer Institute
Keywords: cancer studies; medical imaging; diagnostic accuracy; balanced incomplete block designs; kappa statistics

In multi-reader multi-case cancer imaging studies, trained experts, or readers, review imaging data for each patient independently of one another and characterize the tumor according to one or more features. An intuitive approach is to have each participating reader review all cases, which is often burdensome. An alternative approach to assigning readers to cases that reduces the reading load while still guaranteeing estimability of pairwise metrics (e.g. kappa statistics) for any two readers is a Balanced Incomplete Block (BIB) Design. Under this design, readers only review a fraction of all cases, each reader reviews approximately the same number of cases, and any pair of readers has at least one case in common. Theoretical properties of true and false positive rates (TPR and FPR) and kappa statistics under a BIB Design are derived. Simulation studies are performed to gain further insight into these properties as a function of design and model parameters. It is shown that a 25% reduction in reading load under a BIB Design often induces less than a 5% increase in the standard errors of TPR and FPR and less than a 10% increase in those of kappa statistics.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program