Statistical Models for Reliable Change

• Activity Number: 621
• Type: Contributed
• Date/Time: Thursday, August 2, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Statistics in Mental Health Research
• Author(s): Joseph Olsen*+
• Companies: Brigham Young University
• Keywords: reliable change ; practice effects ; method comparison ; observer agreement ; evidence-based practice

Abstract

In evidence-based clinical practice, continuing interest in the statistical dependability and clinical trustworthiness of test score changes has led to the development of more than a dozen different indices for assessing reliable change (RC) in individual client functioning, where each such index has been presented with its own justification and purported advantages. Each of these RC indices are formulated as the ratio of a designated change estimate to a selected measure of variability or standard error, where both are derived from the bivariate joint distribution of the test and retest scores. This paper develops and applies a general and comprehensive statistical framework for organizing, comparing, and evaluating these various RC indices, and outlines the conditions under which certain of them become statistically equivalent. This framework draws on fundamental statistical principles derived from observer agreement and method comparison studies in medicine. Adjustments for possible general and differential practice effects in serial testing are highlighted. Recommendations for statistical and clinical practice are given.