Quantifying an Agreement Study

• Activity Number: 687
• Type: Contributed
• Date/Time: Thursday, August 8, 2013 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Jason Liao*+
• Companies: Novartis Pharmaceuticals Corporation
• Keywords: agreement ; concordance ; discordance rate ; tolerance probability

Abstract

In medical and other related sciences, clinical or experimental measurements usually serve as a basis for diagnostic, prognostic, therapeutic, and performance evaluations. Measurements of agreement are needed for assessing the reliability of multiple raters (or measurement methods), assessing the reliability of the inclusion criteria for entry into a randomized clinical trial (RCT), validating surrogate endpoints in a study, determining that the important outcome measurements are interchangeable among the evaluators in an RCT. Any elegant clinical trial design cannot overcome the damage by unreliable measurement (Fleiss, 1986). A good measurement agreement is very important and crucial for a clinical trialist. Many methods have been developed to assess the agreement of two measurement methods. However, no method exists to quantify how good the agreement of two measurement methods is. In this talk, similar to the type I error and the power in describing a hypothesis testing, we propose quantifying an agreement assessment using two rates: the discordance rate and the tolerance probability. This approach is demonstrated through examples.