|
Activity Number:
|
240
- Distributions and Significance
|
|
Type:
|
Contributed
|
|
Date/Time:
|
Monday, July 30, 2018 : 2:00 PM to 3:50 PM
|
|
Sponsor:
|
Biometrics Section
|
|
Abstract #330640
|
|
|
Title:
|
The Problems with the Kappa Statistic as a Metric of Inter-Observer Agreement on Lesion Detection Using a Third-Reader Approach When Locations Are Not Pre-Specified
|
|
Author(s):
|
Joanna H Shih* and Matthew D Greer and Baris Turkbey
|
|
Companies:
|
National Cancer Institute and National Cancer Institute and National Cancer Institute
|
|
Keywords:
|
Inter-observer agreement;
Kappa;
Index of specific agreement;
Prostate cancer
|
|
Abstract:
|
Imaging plays an integral role in cancer lesion detection and characterization in oncology practice. For a diagnostic test to gain wide applicability, it is essential to determine its inter-observer agreement. Diagnostic imaging to detect cancer lesions heavily relies on multi-reader studies to measure the inter-observer agreement, and the kappa statistic has been the predominant index of agreement reported in the radiology literature. The purpose of this work is to point out the problem of using kappa to measure reader agreement on lesion detection under the third-reader determined agreement approach and to present two alternative measures of agreement which are more suitable under this setting. The proposed approaches are illustrated using a data set recently collected from a prostate cancer multi-parametric MRI multi-reader study which used the third-reader approach to determine the inter-reader agreement on lesion detection.
|
Authors who are presenting talks have a * after their name.
