|
Abstract:
|
The underlying definition of abnormal behavior via psychopathology has frequently come into question in mental health research (Insel & Cuthbert, 2015). The resulting inconsistency and imprecision in construct definition can markedly reduce the ability to detect mechanistic effects (Kline, 2013). In particular, what is considered acceptable reliability for behavioral data may actually distort research findings substantially (Loken & Gelman, 2017; Lance, Butts, & Michels, 2006).
To evaluate how measurement reliability is affecting clinical research, we assessed the reliability of outcome measures in studies in the NIMH National Database for Clinical Trials Related to Mental Illness, and then calculated how these reliability estimates affected observed effect sizes. Results indicated that many published effect sizes in clinical mental health research are underestimated by 25-40%. Implications for all mental health research will be discussed, along with possible modeling and research design solutions, including latent variable models, statistical disattenuation for unreliability, and principal components-based approaches that utilize multiple informants.
|
