Abstract:
|
In educational testing, an increasingly common issue has been efficient evaluation of a skill by use of multiple sources of information. For example, a writing assessment may include human ratings of essays, electronic essay ratings, and other section scores. Because standard linear regression is not applicable due to unobserved true scores, this paper suggests applications of classical test theory to obtain best linear predictors of composite true test scores based on observed test scores and ancillary data. Such analysis often requires information on repeaters who take the assessment more than once. Because such repeaters are not a random sample of all examinees, adjustment by minimum discriminant information (Haberman, 1984) is applied to reduce the effect of selection bias. Applications are made to TOEFL iBT Writing and another large scale Writing assessment to illustrate the proposed approach. Results obtained indicate that substantial improvements are possible both in terms of reliability of scoring and in terms of assessment reliability.
|
ASA Meetings Department
732 North Washington Street, Alexandria, VA 22314
(703) 684-1221 • meetings@amstat.org
Copyright © American Statistical Association.