Should I Use Gain Scores to Determine Treatment Effect from a Pretest-Posttest Design?: A Simulation Study

• Activity Number: 506
• Type: Topic Contributed
• Date/Time: Wednesday, August 4, 2010 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Education
• Author(s): Michael Posner*+ and AnneMichele Millar and Philip Yates
• Companies: Villanova University and Mount Saint Vincent University and California State Polytechnic University, Pomona
• Keywords: Gain Scores ; Pretest/Posttest ; Survey of Attitudes Towards Statistics ; Monte Carlo Simulation ; Statistics Education ; Gain Scores

Abstract

Pretest-posttest designs improve statistical efficiency by allowing previous data on each subject to serve as self-controls. Typically, gain scores (post minus pre) are used to summarize change, while other measures, like percent increase, are less frequently used. We examine the use of these measures in experimental designs for two treatments, specifically evaluating the effectiveness of classroom interventions. Through simulation, we quantify the bias and precision of various correlations, effect sizes, interaction effects, and missing data structures on gain scores and other alternative measures. This work was motivated by modeling change in student attitudes towards statistics (using the Survey of Attitudes Toward Statistics), which will be discussed as well.