An Exploration of Latent Structure in Process Data

• Activity Number: 424 - Recent Advances in Educational and Psychological Data Analysis
• Type: Invited
• Date/Time: Thursday, August 6, 2020 : 10:00 AM to 11:50 AM
• Sponsor: Journal of Educational and Behavioral Statistics
• Author(s): Jingchen Liu* and Xueying Tang and Zhi Wang and Susu Zhang
• Companies: Columbia University, Department of Statistics and and Columbia University, Department of Statistics and Columbia University, Department of Statistics

Abstract

In classic tests, item responses are often expressed as univariate categorical variables. Computer-based tests allow us to track students’ entire problem solving processes through log files. In this case, the response to each item is a time-stamped process containing students’ detailed behaviors and their interaction with the simulated environment. The key questions are whether and how much more information are contained in the detailed response processes additional to the traditional responses (yes/no or partial credits). Furthermore, we also need to develop methods to systematically extract such information from the process responses that often contain a substantial amount of noise. In this talk, I present exploratory analysis of process data in PIAAC and PISA. The results confirm that the process data do contain a significant amount of additional information and they also provide guideline on efficient item design for future studies.