|
Activity Number:
|
523
|
|
Type:
|
Contributed
|
|
Date/Time:
|
Wednesday, August 5, 2009 : 2:00 PM to 3:50 PM
|
|
Sponsor:
|
Section on Nonparametric Statistics
|
| Abstract - #304022 |
|
Title:
|
A New Permutation-Based Method for the Evaluation of the Agreement Between Two Observers with Replicated Binary Observations
|
|
Author(s):
|
Yi Pan*+ and Michael J. Haber
|
|
Companies:
|
Emory University and Emory University
|
|
Address:
|
1518 Clifton Rd, Atlanta, GA, 30322,
|
|
Keywords:
|
Agreement ; Equivalence ; Coefficient of Individual Agreement ; Binary Ratings
|
|
Abstract:
|
A new coefficient for the evaluation of agreement between observers making replicated binary ratings compares the observed average disagreement to its expected value under the hypothesis of equivalence. Equivalence of two observers requires that for each study subject the conditional distributions of the scores of the two observers are identical. The expected disagreement under equivalence for a subject is based on (all possible) allocations of all the observations made on the subject between the two observers. A simple method for estimating the standard error of the new coefficient is also presented. The new coefficient is compared with the coefficient of individual agreement (CIA), which is based on comparing the inter- and intra-observer disagreements. Data from a study involving the evaluation of mammograms by ten radiologists is used to illustrate this new approach.
|
