A Case of Erasure Fraud in a Cincinnati School: The Role of Statistical Forensics

• Activity Number: 453
• Type: Topic Contributed
• Date/Time: Wednesday, August 1, 2012 : 8:30 AM to 10:20 AM
• Sponsor: Section on Government Statistics
• Author(s): Martin Levy*+
• Companies: University of Cincinnati
• Keywords: cheating ; control chart ; p-value ; erasure analysis

Abstract

In the fall of 2010, the Ohio Department of Education (ODOE) accused a Cincinnati school of erasure fraud based upon infrared scans of standardized test papers. Using control chart-like methodology, ODOE found that in several classes, the number of wrong to right erasures on standardized exams exceeded the state average by as many as 14 standard deviations. More devastating was the accusation of a pattern evidently produced by "an incompetent mastermind" in 5th grade science where there were several questions with nearly unanimous conversions of right to wrong answers! Two statistical lines of defense were used. The first was based on the fact that the teachers had instructed the students in a method called strategic test taking, i.e., "when in doubt, mark two and then erase one" before exam completion. A simulation tool was developed to demonstrate the effects of the strategy on the deviation of their results from those of the state. The second was a set of probability calculations designed to show that the 5th grade science patterns did not rise to a high level of improbability using a p-value-type argument. The two sides eventually settled to their mutual satisfaction.