New Methods in Item Response Theory: Information, Bias, and Mean Square Error

• Activity Number: 473 - Tools of Inferential Decision Making in Education and Behavioral Sciences
• Type: Contributed
• Date/Time: Wednesday, August 2, 2017 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Bivin Sadler*
• Companies: Southern Methodist University

Abstract

Item response theory (IRT) is widely used to model a subject's probability of a correct response at the item level and hence provides a methodology for obtaining MLEs of the subject's latent ability. Since analysis using IRT occurs at the item level, this research shows that the selection of items that is optimal (in terms of MSE) in estimating individual ability is not necessarily the same set of items that is optimal when estimating mean ability. A theoretical explanation of this behavior is provided using a paper from Efron and Hinkely (1978.) As a result of this research, two new methods were developed. The first provides a calculation of the true information, bias and thus MSE for tests that estimate both individual and mean ability; although, is limited to tests of 15 items or less. The second is found to be best used for estimating information, bias and MSE for larger tests (more than 15 items) whose goal is to estimate mean ability. A final procedure is introduced that uses the bias calculation/approximation methods to correct for bias in the MLEs. Examples involving standard errors and test construction are used throughout the paper to explain and demonstrate the theory and methods.