Optimal Design for Interobserver Agreement Studies

• Activity Number: 226
• Type: Contributed
• Date/Time: Tuesday, August 10, 2004 : 10:30 AM to 12:20 PM
• Sponsor: ENAR
• Author(s): Mekibib Altaye*+
• Companies: Cincinnati Children's Hospital Medical Center
• Keywords: power ; interobserver ; optimal ; cost ; agreement

Abstract

Power considerations are clearly important in choosing the number of subjects and raters that should be recruited for a study of interobserver agreement. However, most studies are also subject to budget constraints, requiring that the investigator achieve specified power while minimizing the total study cost C. This cost will typically depend on C1, the cost of recruiting a subject (n), C2, the cost of recruiting a rater (m) and C3, the cost of obtaining a measurement on a subject that has already been recruited. We introduce a linear cost function from which the optimal number of raters and subjects can be determined. The optimal values will be derived under the condition that the total cost for sampling and measurement is fixed at C, with the variance of the kappa statistics minimized subject to the restriction on cost given by the proposed model. The results of the minimization process indicate that for a fixed total cost C the effect of varying the allocation of costs among raters, subjects and measurements impacts the required number of subjects much more than varying the number of raters.