An Overview of Different Methods to Assess Agreement in Early Phase Clinical Trials

• Activity Number: 398
• Type: Topic Contributed
• Date/Time: Tuesday, August 2, 2011 : 2:00 PM to 3:50 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Radha Railkar*+ and Richard Baumgartner and Dai Feng and Cynthia Gargano and Patrick Larson and Lori Mixson
• Companies: Merck & Co., Inc. and Merck & Co., Inc. and Merck & Co., Inc. and Merck & Co., Inc. and Merck & Co., Inc. and Merck & Co., Inc.
• Keywords: agreement ; biomarker ; intraclass correlation coefficient ; concordance correlation coefficient ; within subject coefficient of variation

Abstract

Early phase clinical trials are often undertaken to evaluate new measurement techniques or biomarkers that could help to rapidly identify clinical target activity of new compounds with smaller numbers of patients/subjects. In these studies it is important to establish that the new measurement technique or biomarker agrees with an existing gold standard or that 2 or more repeat measurements obtained using the new technique or biomarker, agree with one another (i.e., the technique or biomarker is repeatable). Common methods to assess agreement include the intraclass correlation coefficient (ICC), the concordance correlation coefficient (CCC), the within subject coefficient of variation, and graphical methods such as the Bland-Altman plot. We discuss the pros and cons of the various methods. Additionally, multiple models and methods have been proposed to estimate the ICC and CCC. We propose that estimating the ICC/CCC from a 2-way mixed effects model with the repeat measurements as fixed effects is appropriate for evaluating agreement in small studies. Simulation studies were performed to compare the different estimation methods for ICC/CCC and our recommendations are presented.