Abstract:
|
A simulation study was conducted to explore the performance of the linear logistic test model (LLTM) when the relationships between items and cognitive components were misspecified. Marginal maximum likelihood and empirical Bayes were used in SAS to estimate parameters for cognitive components and item difficulty, as well as person ability, respectively. Factors manipulated in the simulation study included overall type of misspecification (under-, balanced-, and over-specification), percent misspecified (1%, 5%, 10%, and 15%), sample size (20, 40, 80, 160, 320, 640, and 1280), Q-matrix density (64% and 46%), skewness of ability distribution (-0.5, 0, and 0.5), and test length (20, 40, and 60 items). Statistical bias, RMSE, CI coverage, and CI width were computed for estimates across the set of replications. As the Q-Matrix moved away from the truth, cognitive components and item difficulty estimates became progressively more biased (positive bias with under-specification, and negative bias for balanced- and over-specification). Results were interpreted for the design factors and recommendations for the application of LLTM in assessment studies were provided.
|
Copyright © American Statistical Association.