Evaluating the Sample Invariance Property of the Standard Error of Measurement

• Activity Number: 398
• Type: Contributed
• Date/Time: Wednesday, August 10, 2005 : 10:30 AM to 12:20 PM
• Sponsor: Biopharmaceutical Section
• Author(s): Samiran Ghosh*+ and Joseph Cappelleri and Andrew Bushmakin and William R. Lenderking
• Companies: University of Connecticut and Pfizer, Inc. and Pfizer, Inc. and Pfizer, Inc.
• Keywords: Classical Test Theory ; Health Services Research ; Psychometrics ; Standard Error of Measurement ; Schwartz Outcomes Scale (SOS-10)

Abstract

Under classical test theory (CTT), it often is claimed that the standard error of measurement (SEM) of a measurement scale is sample invariant [SEM = standard deviation*square root of (1 - scale reliability)]. To evaluate this claim, we conducted Monte Carlo simulations on real and simulated data. The real data were obtained from a study of obsessive-compulsive disorder that used the Schwartz Outcomes Scale (SOS-10), a measure of the effectiveness of psychiatric treatment. The simulations involved random samples of varying size in which the standard deviation and Cronbach's alpha (\alpha, a measure of scale reliability) were computed for 300,000 replications. The invariance property was tested based on a theoretical regression model [SD^2 = SEM^2/(1 - \alpha)] in which the regression coefficient SEM^2 and the coefficient of determination (r^2) were estimated. The estimated SEM^2 was 12.19 (close to the true value of 12.41), and the estimated r^2 of 0.99 was virtually perfect (and close to the theoretical value of 1). Therefore, results based on clinical trial data were confirmed with simulated data. The claim that the SEM of a measurement scale is sample invariant is supported.