Online Program Home
My Program

Abstract Details

Activity Number: 294 - SPEED: Statistical Learning and Data Science Speed Session 2, Part 1
Type: Contributed
Date/Time: Tuesday, July 30, 2019 : 8:30 AM to 10:20 AM
Sponsor: Section on Statistical Learning and Data Science
Abstract #306929 Presentation
Title: Stacked Ensemble Learning for Propensity Score Methods in Observational Studies
Author(s): Maximilian Autenrieth* and Richard Levine and Juanjuan Fan and Maureen Guarcello
Companies: San Diego State University and Ulm University and San Diego State University and San Diego State University and San Diego State University
Keywords: propensity score; machine learning; stacked generalization - ensemble learning; inverse probability of treatment weighting; observational studies; educational studies

Propensity score (PS) methods have shown to reduce selection bias in observational studies. We introduce a stacked generalization ensemble learning approach to improve propensity score estimation by fitting a meta learner on the predictions of a suitable set of diverse base learners. We perform a comprehensive Monte Carlo simulation study, implementing eight scenarios that mimic characteristics of typical data sets in educational studies. Our proposed ensembles led to superior reduction of bias compared to the current state-of-the-art in PS estimation. Further, our simulations imply that common used balance measures (ASAM) might be misleading as PS model selection criteria. Our findings suggest that a combination of stacked ensembles will allow educational researchers to obtain more precise treatment effect estimates in propensity score studies. We apply our best models to assess the average treatment effect of a Supplemental Instruction (SI) program in an introductory psychology course at San Diego State University and confirm results in the recent literature that SI has a significant positive impact on student success.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program