Scoring and then analyzing or analyzing while scoring: An application of GLMM to an education instrument development and analysis

• Activity Number: 302
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Mark Greenwood*+ and Dan Jesse
• Companies: Montana State University and RMC Research Corporation
• Keywords: GLMM ; Rasch model ; Longitudinal data ; Mixed models ; Scoring

Abstract

Psychometric methods provide clear ideas for mapping scored (binary) items into overall test scores using Rasch or Item Response Theory models or, more simply, scores can be based on percentages. We explore potential analysis techniques for longitudinal responses when large, randomly sampled test data sets may not be available, merging the scoring and analysis processes into a single model. Specifically, logistic generalized linear mixed models are used to analyze the binary item responses for scoring information (item difficulty and subject scores) and to address longitudinal research questions in the same model. This approach is compared to a generalized linear mixed model analysis of the count of items correct and a linear mixed model analysis of the one-parameter logistic model score results based on an independent data set. The different methods are motivated by the development and analysis of an instrument from a five-year longitudinal study of coaches of K-8 mathematics teachers. In this application, there are negligible differences in using the three different approaches.