Diagnostic Accuracy Evaluation of Diagnostic Assessment Model in Longitudinal Data: a Simulation Study of Neural Network Approach

• Activity Number: 353 - SPEED: Statistical Learning and Data Science Speed Session 2, Part 2
• Type: Contributed
• Date/Time: Tuesday, July 30, 2019 : 10:30 AM to 11:15 AM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Chi Chang* and Harlan McCaffery
• Companies: Michigan State University and University of Michigan
• Keywords: Neural Network; Diagnostic Accuracy; Formative Assessment; Diagnostic Assessment Model; Longitudinal Study; Attribute Patterns

Abstract

Studies applying the neural network algorithm in the formative assessment settings are very few. The purpose of this research is to evaluate the diagnostic accuracy of the diagnostic assessment model in longitudinal settings with neural network algorithms. Different levels of three study factors were investigated in the simulation study: 1) the number of time points: 2, 3, and 4, 2) the number of items assessed each time: 30 and 60, 3) the number of subjects taking the test: 500 and 1000. The number of attribute patterns is fixed at eight, and the distribution of the attributes in items are fixed to be 50% of items possess a single attribute, 30% possess two attributes, and 20% possess three attributes. We combined the latent transition modeling and DINA model as the longitudinal diagnostic assessment model to evaluate the longitudinal data. The neural network approach is integrated in the learning algorithm. 500 replications were used in each condition. Diagnostic accuracy in each condition were evaluated.