Bayesian Joint Item Response Model of Multidimensional Response Data with Application to Computerized Testing

• Activity Number: 380 - Advances in Statistical Learning for Large-Scale and Multidimensional Testing Data
• Type: Topic-Contributed
• Date/Time: Thursday, August 12, 2021 : 12:00 PM to 1:50 PM
• Sponsor: International Chinese Statistical Association
• Author(s): Xiaojing Wang* and Fang Liu and Ming-Hui Chen and Roeland Hancock
• Companies: University of Connecticut and Northeast Normal University and UCONN and University of Connecticut
• Keywords: Computerized tests; DIC decomposition; IRT models; LPML decomposition; Pencil-and-paper tests; Response times

Abstract

Computerized assessment provides rich multidimensional data including trial-by-trial accuracy and response time measures. A key question in modeling this data is how to incorporate response time data, for example, in aid of ability estimation in item response theory (IRT) models. To address this, we propose two new model comparison criteria based on the decomposition of deviance information criterion (DIC) and the logarithm of the pseudo-marginal likelihood (LPML). The proposed criteria can quantify the improvement on the fit of item responses due to incorporating the response time (and standard scores from pencil-and-paper tests) in a conjoint item response model. Simulation studies are conducted to examine the empirical performance of the proposed model selection criteria, and these approaches are illustrated on a real dataset from a computerized educational assessment program. In the real analysis, we also put forward some novel ideas to rank the item difficulty with uncertainty and examine the residuals of the proposed joint model for model adequacy.