|
Abstract:
|
Statistical process control is an important business strategy for many organizations (Montgomery, 2009). The testing industry is no exception. For operational educational assessments, it is common to reuse test questions, or items, in different test administrations throughout the life of the assessments to allow test scores across administrations to be made comparable. A typical assumption is that a reused item should perform relatively similarly over time. In reality, one possible issue with item reuse is that an item may become easier with exposure (i.e., cheating occurs). Thus, it is desirable to continuously monitor the performance of the reused items to detect if cheating occurs. CUSUM analysis (Page, 1954) is one of the popular quality control procedures applied to detect and diagnose changes in a process over time. In this paper, the upper one-sided CUSUM statistic is employed to monitor the residual between the observed and the predicted percents of correct responses based on models commonly used in psychometrics by test-taker group. The approach is found effective in detecting cheating with group size as small as 10.
|
