Assessing Agreement: A Graphical Approach

• Activity Number: 372
• Type: Contributed
• Date/Time: Tuesday, August 11, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Paul Hshieh* and Tie-Hua Ng
• Companies: CBER/FDA and FDA/CBER
• Keywords: Bland-Altman plot, systematic bias ; Q-Q plot, variability ; graphical approach

Abstract

Altman and Bland (1983) criticized the use of correlation, regression and the mean difference in the comparison of two methods of measurement. They proposed to plot the difference between the two methods against the average (Bland-Altman plot). A scatter plot and Q-Q plot are useful graphical tools to assess the agreement of two methods of measurement.

In this paper, we proposed a new "procedure" in assessing the agreement of a new method to an old method using three graphical approaches: scatter plot, Bland-Altman plot and Q-Q plot. In this procedure, subjects are measured twice using the Old method (Old1, Old2) and once using the New method (New). In each graphical approach, the plot of Old1 vs. Old2 is used as a norm in which the plots of New vs. Old1 and New vs. Old2 can be compared with. If the new method is comparable to the old, then the plots of New vs. Old1 and New vs. Old2 should each appear similar to the plots of Old1 vs. Old2.

A simple measurement error model is used in a simulation study to learn how the systematic bias and unequal variance are revealed in these plots. Such learning can then be used to assess the agreement of two methods in real life. A real example is used to illustrate the new "procedure".

Key Words: Bland-Altman plot, Q-Q plot, graphical approach, systematic bias.