Thinking Critically About Undergraduate Testing

• Activity Number: 109
• Type: Invited
• Date/Time: Monday, August 1, 2011 : 8:30 AM to 10:20 AM
• Sponsor: IMS
• Author(s): Eric Loken*+
• Companies: Penn State University
• Keywords: Teaching ; Measurement ; Testing

Abstract

At many large universities, science, technology, engineering, and mathematics (STEM) departments function like major testing organizations, delivering hundreds of thousands of multiple choice tests. And yet departments rarely dedicate any resources to the construction, analysis or evaluation of tests. This is surprising considering risks in accountability, and lost opportunities for innovation in pedagogy. Multiple choice tests are often guaranteed to provide unequal information across the ability spectrum, and almost nothing is known about the consistency of measurement properties across subgroups. Course management systems that encourage testing from item banks can expose individual students to dramatically unequal assessment. Aside from issues of fairness and validity, the neglect of research on testing in undergraduate classes represents a missed opportunity to take an empirical approach to pedagogy. Years of testing have generated vast amounts of data on student performance. These data can be leveraged to inform pedagological approaches. They can also be leveraged to provide novel assessments and tools to better encourage and measure student learning.