|
Activity Number:
|
247
|
|
Type:
|
Contributed
|
|
Date/Time:
|
Tuesday, August 8, 2006 : 8:30 AM to 10:20 AM
|
|
Sponsor:
|
Biometrics Section
|
| Abstract - #306031 |
|
Title:
|
Assessing Rater Exchangeability and Identifying an Atypical Rater Using a Log-linear Modeling Approach
|
|
Author(s):
|
Kari Kastango*+ and Roslyn A. Stone
|
|
Companies:
|
University of Pittsburgh and Veteran's Affairs Pittsburgh Healthcare System
|
|
Address:
|
Graduate School of Public Health, Pittsburgh, PA, 15261,
|
|
Keywords:
|
inter-rater agreement ; nominal outcome
|
|
Abstract:
|
We propose a formal inferential approach to identify an atypical rater in an inter-rater agreement study involving six raters and a binary outcome. A global and heterogeneous partial agreement log linear model is fit and pair-wise comparisons of the six partial agreement parameters are made, using p-values adjusted for the number of comparisons made. Type I error and the power to correctly identify an atypical rater are assessed via simulation. The heterogeneous partial agreement parameters generally do highlight the most atypical rater in the scenarios considered, although the power is low to detect such a rater as statistically significantly different from the other raters. The pair-wise comparisons of the heterogeneous partial agreement parameters are quite likely to identify the correct rater as atypical when any rater is identified.
|
