|
Abstract:
|
In multi-reader multi-case cancer imaging studies, trained experts, or readers, review imaging data for each patient independently of one another and characterize the tumor according to one or more features. An intuitive approach is to have each participating reader review all cases, which is often burdensome. An alternative approach to assigning readers to cases that reduces the reading load while still guaranteeing estimability of pairwise metrics (e.g. kappa statistics) for any two readers is a Balanced Incomplete Block (BIB) Design. Under this design, readers only review a fraction of all cases, each reader reviews approximately the same number of cases, and any pair of readers has at least one case in common. Theoretical properties of true and false positive rates (TPR and FPR) and kappa statistics under a BIB Design are derived. Simulation studies are performed to gain further insight into these properties as a function of design and model parameters. It is shown that a 25% reduction in reading load under a BIB Design often induces less than a 5% increase in the standard errors of TPR and FPR and less than a 10% increase in those of kappa statistics.
|
