|
Abstract:
|
We discuss two different contexts for measuring statistical agreement for continuous variables. Existing literature on statistical agreement focuses almost exclusively on the symmetric context, measuring how interchangeable two methods are. More useful in our experience is the predictive context, measuring how well a test method predicts a reference method - e.g. how well estimates using observational controls match those from an A/B experiment. A widely used agreement coefficient in the symmetric context is Lin's (1989) concordance correlation coefficient (CCC). We propose a counterpart of the CCC for use in the predictive context, which we call the predictive correlation coefficient (PCC). Like CCC, PCC is a product of a precision component (Pearson correlation) and an accuracy component. The main distinction is that the accuracy component of the CCC measures the marginal similarity between two measurements x and y, while the accuracy component of PCC measures the marginal similarity between x and the regression of y on x. We present applications of the use of both CCC and PCC for evaluating comparability of methods for Google's Brand Lift product.
|
