Online Program

Return to main conference page

Wednesday, January 10
Wed, Jan 10, 5:30 PM - 7:00 PM
Crystal Ballroom CD & Prefunction
Welcome Reception & Poster Session I

Comparing the performance of methods that generalize from small samples to large target populations (304259)

Cyrus Ebnesajjad, Johns Hopkins School of Public Health 
*Ian Schmid, Johns Hopkins School of Public Health 
Elizabeth A Stuart, Johns Hopkins Bloomberg School of Public Health 

Keywords: generalizability, sample selection model, small sample, target population, simulation, weighting, machine learning

Generalizing a sample average treatment effect (SATE) to a target population requires fitting a model of sample selection similar in spirit to treatment assignment models used for propensity scores. When the size of the experimental sample is small and the target population is large, the estimated sample selection model, and thus the estimated target average treatment effect (TATE), may be susceptible to biases due to class imbalance and the small sample size. Various parametric and nonparametric methods including logistic regression, generalized boosting, and Bayesian Additive Regression Trees are available to address these issues. We compare their performance in generalizing the SATE to the TATE in complex simulated data according to bias, mean squared error, and 95% confidence interval coverage rate. We vary the proportion of the population sampled, the total population sample size, and the complexity of the selection model.