Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 277 - ASA Student Paper Competition Winners
Type: Topic-Contributed
Date/Time: Wednesday, August 11, 2021 : 1:30 PM to 3:20 PM
Sponsor: Government Statistics Section
Abstract #317604
Title: Varying Impacts of Letters of Recommendation on College Admissions: Approximate Balancing Weights for Subgroup Effects in Observational Studies
Author(s): Eli Ben-Michael* and Avi Feller and Jesse Rothstein
Companies: Harvard University and UC Berkeley and UC Berkeley
Keywords: Causal Inference; Observational Studies
Abstract:

In a pilot study during the 2016-17 admissions cycle, the University of California, Berkeley invited many applicants for freshman admission to submit letters of recommendation. We are interested in estimating how impacts vary for under-represented applicants and applicants with differing a priori probability of admission. Assessing treatment effect variation in observational studies is challenging, however, because differences in estimated impacts across subgroups reflect both differences in impacts and differences in covariate balance. To address this, we develop balancing weights that directly optimize for ''local balance'' within subgroups while maintaining global covariate balance between treated and control populations. We then show that this approach has a dual representation as a form of inverse propensity score weighting with a hierarchical propensity score model. In the UC Berkeley pilot study, our proposed approach yields excellent local and global balance, unlike more traditional weighting methods, which fail to balance covariates within subgroups. We find that the impact of letters of recommendation increases with the predicted probability of admission.


Authors who are presenting talks have a * after their name.

Back to the full JSM 2021 program