Exploration of Latent Structure in Test Review and Revision Log Data

• Activity Number: 417 - Statistical Methods for Discovering Latent Structures in High-Dimensional and Complex Data
• Type: Topic Contributed
• Date/Time: Wednesday, August 10, 2022 : 10:30 AM to 12:20 PM
• Sponsor: Section on Bayesian Statistical Science
• Author(s): Susu Zhang* and Anqi Li and Shiyu Wang
• Companies: University of Illinois at Urbana-Champaign and University of Illinois at Urbana-Champaign and University of Georgia
• Keywords: process data; psychometrics; cluster analysis

Abstract

In computer-based tests allowing revision and reviews, examinees' sequence of visits and answer changes to questions can be recorded. These variable-length revision log data introduce new complexities to the observed data but, at the same time, provide additional information on examinees' test-taking behavior, which may inform test development and instructions. The current study provides an exploratory analysis of the item-level revision and review log data via sequence feature extraction and clustering. Based on the revision log data collected from several computer-based assessments, common prototypes of revisit and review behavior are identified. Relationship between revision behavior and various item, test, and individual covariates is further studied under a multivariate generalized linear mixed model.