Making Computerized Adaptive Testing a Diagnostic Tool

• Activity Number: 399
• Type: Topic Contributed
• Date/Time: Tuesday, August 2, 2011 : 2:00 PM to 3:50 PM
• Sponsor: ENAR
• Author(s): Hua-Hua Chang*+
• Companies: University of Illinois at Urbana-Champaign
• Keywords: sequential design ; computerized adaptive testing ; cognitive diagnosis ; latent trait estimation ; latent class classification ; educational testing

Abstract

Computerized adaptive testing (CAT) has become popular in many high-stakes educational testing programs. The goal of a traditional CAT item selection algorithm is to sequentially select items that optimize the estimation process for the examinee's latent trait theta. As a result, only a total score is reported. A new design of CAT also provides feedback about students' individual educational needs in addition to the single overall score. In fact, the US government requires that such diagnostic feedback be provided to parents, teachers, and students as soon as practical. In this paper new methods of classifying examinees into a set of partially ordered latent classes are introduced. The results of both simulation study and a field test of several thousand students will be presented. Issues in estimation accuracy and classification consistency will be addressed. Our research is showing that the proposed Cognitive Diagnostic CAT can be used effectively not only to estimate an examinee's latent trait, but also to classify the examinee's mastery levels of the skills the test is designed to measure. CAT is revolutionarily changing the way we address challenges in assessment and learning.