Comparing Students' Performance Profiles Between Performance-Based and Multiple-Choice Tests Using Multivariate Approach

• Activity Number: 519
• Type: Contributed
• Date/Time: Wednesday, August 6, 2014 : 10:30 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Weiwei Cui*+
• Companies: NISS
• Keywords: performance-based test ; online research and comprehension assessment ; multivariate approach

Abstract

The Online Research and Comprehension Assessment Project developed tests to measure 1800 7th graders' abilities and strategies for science-related online research. Tests were administered in two ways: with access to a pseudo-internet with over 400 imported websites and an internal search engine, and with no internet access as a multiple-choice on-screen test using substitute prepared screenshots. Testing was replicated on a second day when each student took a second version of the test in the alternate format. When complex skill domains are involved, as for many performance-based tests, the key question in modeling is to address the multiple skills both collectively as they are integrated and individually. This performance-based test requires students to make linkages between segments of the test; the multiple choice test does not. Thus, the two types of tests have different levels of multidimensionality. A multivariate approach was taken to model and analyze students' performance profiles. Profile differences between types of tests were analyzed on a within-student basis and then modeled as a function of external factors, students' attributes, and other performance measures.