Consequences of Ignoring Clustering of Item Responses Obtained from Complex Samples During Item Response Theory Scaling

• Activity Number: 560
• Type: Contributed
• Date/Time: Thursday, August 6, 2009 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Aaron Douglas*+
• Companies: Mathematica Policy Research, Inc.
• Keywords: Item Response Theory ; Factor Analysis ; Multilevel IRT ; Multilevel Factor Analysis ; Latent Trait Models ; Clustered Samples

Abstract

This investigation demonstrates how ignoring clustering of item responses results in biased estimation of item discrimination parameters, and biased test information functions, for the two-parameter normal ogive model. Mplus was used to simulate 100 replications of item response data for a 20-item test based on a multilevel data structure of students clustered within schools. Three models were then fitted to the simulated data: 1) a multilevel two-parameter normal ogive model that had the same specification as the data-generating model; 2) a student-level two-parameter normal ogive model that applied Taylor Series expansion for computation of cluster-robust standard errors; and 3) the standard two-parameter normal ogive model. The models were compared to investigate recovery for true values of item parameters, and theoretical consistency for test information, at the student level.