Online Program Home
  My Program

All Times EDT

Abstract Details

Activity Number: 164 - Social Statistics Speed Session
Type: Contributed
Date/Time: Tuesday, August 10, 2021 : 10:00 AM to 11:50 AM
Sponsor: Social Statistics Section
Abstract #318663
Title: Bayesian Estimation of Program-Specific Impacts in the HPOG Program
Author(s): Stas Kolenikov* and David Ross Judkins
Companies: Abt Associates and Abt Associates
Keywords: heterogeneous treatment effects; small area estimation; hieararchical Bayesian modeling; generalized linear mixed models; impact evaluation; labor force training

Health Profession Opportunity Grants (HPOG) programs funded by the HHS ACF provide career transition support for low-income adults. We report the methods used in an evaluation to assess the success of these HPOG programs and to provide local, program-specific estimates of impact. Direct estimation of the local average treatment effect consists of simply comparing the means of outcomes for the local treatment and control groups. To overcome small sample sizes typical for most programs, we developed a complementary set of Bayesian estimates of local impacts based on mixed effect models with program-level random effects and random slopes for the treatment indicator. Before preparing Bayesian estimates of local program effects for the second round of grants (HPOG 2.0), we demonstrated the techniques on the previous round (HPOG 1.0). In addition to allowing methods revisions free of p-hacking, we were able to use the posterior distributions for variance components from HPOG 1.0 as priors for HPOG 2.0. Bayesian estimates of impact exhibited less variability than direct estimates did. Bayesian credible intervals were shorter than the confidence intervals at the same coverage level.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2021 program