Online Program Home
My Program

Abstract Details

Activity Number: 28 - SPEED: A Mixture of Topics in Health, Computing, and Imaging
Type: Contributed
Date/Time: Sunday, July 29, 2018 : 2:00 PM to 3:50 PM
Sponsor: Mental Health Statistics Section
Abstract #329327
Title: Measurement Reliability in Mental Health Research: Critical Implications for Research Design and Analysis
Author(s): Alessandro De Nadai* and Marieke Visser
Companies: Texas State University and Texas State University
Keywords: Reliability; Power; Clinical Research

The underlying definition of abnormal behavior via psychopathology has frequently come into question in mental health research (Insel & Cuthbert, 2015). The resulting inconsistency and imprecision in construct definition can markedly reduce the ability to detect mechanistic effects (Kline, 2013). In particular, what is considered acceptable reliability for behavioral data may actually distort research findings substantially (Loken & Gelman, 2017; Lance, Butts, & Michels, 2006).

To evaluate how measurement reliability is affecting clinical research, we assessed the reliability of outcome measures in studies in the NIMH National Database for Clinical Trials Related to Mental Illness, and then calculated how these reliability estimates affected observed effect sizes. Results indicated that many published effect sizes in clinical mental health research are underestimated by 25-40%. Implications for all mental health research will be discussed, along with possible modeling and research design solutions, including latent variable models, statistical disattenuation for unreliability, and principal components-based approaches that utilize multiple informants.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program