|
Activity Number:
|
16
- Assessing Agreement and Reader Reliability in Medical Imaging Analysis
|
|
Type:
|
Topic Contributed
|
|
Date/Time:
|
Sunday, August 7, 2022 : 2:00 PM to 3:50 PM
|
|
Sponsor:
|
Biopharmaceutical Section
|
|
Abstract #322400
|
|
|
Title:
|
Limits of Agreement for an MRMC Study
|
|
Author(s):
|
Si Wen*
|
|
Companies:
|
Division of Imaging, Diagnostics and Software Reliability, FDA/CDRH/OSEL
|
|
Keywords:
|
Limits of agreement;
MRMC study;
arbitrary study design
|
|
Abstract:
|
A multi-reader multi-case (MRMC) analysis is usually applied to account for both case and reader variability when evaluating the clinical performance of a medical imaging device or reader performance under different reading modalities. For a clinical task that measures a quantitative biomarker, an agreement analysis such as the limits of agreement (LOA) method can be used to assess the performance of the readers. To decompose the sum of squares, a three-way mixed effect ANOVA model is applied to estimate the variance of individual differences and the LOA between different reading modalities. There are rules for the expected mean squares for fully-crossed data, where all the readers read all the cases in all modalities being studied. But sometimes the annotation task is labor-intensive and time-consuming, so that a fully-crossed study is not practical. In this work, we focus on limits of agreement analysis for an MRMC study with an arbitrary study design. Simulation studies were conducted to validate the LOA variance estimates. The method was also applied to compare pathologist performance for assessing the density of stromal tumour infiltrating lymphocytes on different platforms.
|
Authors who are presenting talks have a * after their name.
Back to the full JSM 2022 program
|
