Assessing Goodness of Fit of Item Response Theory Models Using Generalized Residuals

• Activity Number: 176
• Type: Contributed
• Date/Time: Monday, August 1, 2011 : 10:30 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Sandip Sinharay*+ and Shelby J. Haberman
• Companies: ETS and ETS
• Keywords: model fit ; psychometrics ; educational statistics ; item response theory

Abstract

Item response theory (IRT) models (e.g., Lord, 1980), which are latent structure models in which manifest variables are polytomous and the latent variable/vector is polytomous or continuous, are often applied to scores on questions/items on standardized achievement or aptitude tests (Junker, 1993). Despite their long history, it is often not obvious how to rigorously assess the discrepancies between such models and observed data (Hambleton & Han, 2005). This situation exists despite the fact that Standard 3.9 of the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council for Measurement in Education, 1999) demands evidence of model fit when an IRT model is used to make inferences from a test data set. Generalized residuals are a tool employed in the analysis of contingency tables to examine goodness of fit. The essential feature of these residuals is that a linear combination of observed frequencies is compared to its estimated expected value under a proposed model. We apply these residuals to IRT models. Their use is illustrated with data from operational testing programs.