Using Signal Detection Theory to Verify Performance Standard Setting on Michigan English Language Proficiency Assessment

• Activity Number: 629
• Type: Contributed
• Date/Time: Thursday, August 4, 2011 : 8:30 AM to 10:20 AM
• Sponsor: Social Statistics Section
• Author(s): Ji Zeng*+ and Joseph A. Martineau
• Companies: Michigan Department of Education and Michigan Department of Education
• Keywords: signal detection theory ; English language assessment ; cut scores

Abstract

Recent education policies aim to raise academic expectations for all students. For example, there is an expectation that English Language Learners (ELLs) will move to rigorous main-stream classes as soon as they are ready to succeed in those classrooms. This paper uses signal detection theory with Michigan Merit Exam (MME) Reading proficient as an external criterion to examine the existing proficient cuts at grades 11 on Michigan English Language Proficiency Assessment (ELPA). Moreover, the resulting optimal cut on ELPA for grade 11 was used as the criterion to locate the optimal cut for grade 10, and so on and so forth, until we locate the optimal cut for Kindergarteners. The obtained suggested cuts were shown to be fitted by two separate polynomials of order 2 with regard to grade levels. Moreover, when the adjusted optimal cuts (after a quick fix to join two curves into one) were compared with the adjusted cuts (for fair comparison) set by standard setting panel, it seems that the ELPA standard setting panel was more lenient when setting the proficient cuts for higher grade levels, but was more stringent at the two lowest grade levels.