A Partial Simulation Study of Phantom Effects in Multilevel Analysis of School Effects: The Case of School Socioeconomic Composition

• Activity Number: 337 - SPEED: Methodological Developments in Social Statistics, Part 1
• Type: Contributed
• Date/Time: Tuesday, July 30, 2019 : 10:30 AM to 12:20 PM
• Sponsor: Social Statistics Section
• Author(s): Xin Ma and Hao Zhou*
• Companies: University of Kentucky and University of Kentucky
• Keywords: Multilevel Modeling; Composition Effects; Phantom Effects

Abstract

In the study of school effects, researchers adopt a multilevel model to examine the effects of school socioeconomic status or SES on academic achievement. They typically do not include prior academic achievement, a variable highly related to present academic achievement. In the absence of prior academic achievement, there are effects of school SES on academic achievement, but in the presence of prior academic achievement, the effects of school SES on academic achievement tend to disappear. This is defined as phantom effects. We conducted a partial simulation study based on 2015 Program for International Student Assessment (PISA). In a multilevel model with students nested within schools, we created prior measures of science achievement with correlation as .05, .15, .25, .35, .45, .55, .65, .75, .85, and .95 with the PISA measure of science achievement. Each prior measure is normally distributed and share the same measurement scale as the PISA measure. After using each prior measure, we observed the curvilinear decrease and eventual disappearance of the effects of school SES. The phantom effects of school SES appear around .55 as correlation between prior and PISA measures.