Effectiveness of adolescent substance abuse treatments: An application using multinomial propensity scores
Daniel Almirall, University of Michigan, RAND 
Beth Ann Griffin, RAND Corporation 
Daniel McCaffrey, RAND 
Rajeev Ramchand, RAND 
*Megan Schuler, Johns Hopkins Bloomberg School of Public Health, RAND 

Keywords: causal inference, propensity scores, multinomial treatments, adolescent, substance abuse treatment

Objective: The objective of this project was to compare the relative effectiveness of four treatment modalities for adolescent substance treatment: urine drug screening (UDS) only, Motivational Enhancement Therapy-Cognitive Behavioral Therapy (MET-CBT5), MET-CBT5 combined with UDS, and no treatment. Methods: Data is from a multisite, longitudinal observational study of 5186 adolescents enrolled in a variety of substance treatment programs. Adolescents were classified into four groups based on based on self-reported received services. Outcomes of interest were frequency of substance use and severity of substance use problems during follow-up, as measured by the GAIN survey. Multinomial propensity score weighting was used to adjust for pretreatment covariate imbalances between the four treatment groups. Weights were estimated using Generalized Boosted Models (GBM) and we discuss rules for tuning the GBM model to yield good balance. Weighted regression analyses estimate the impact of treatment on each outcome. Results: Propensity score weights resulted in comparable treatment groups. Urine drug screening was associated with a significant decrease in both the frequency of substance use and of substance use problems through 12 months of follow-up. Although MET-CBT5 showed a trend of reducing frequency of substance use, no statistically significant evidence was observed. Combining UDS and MET-CBT5 does not appear to result in more favorable outcomes than UDS alone. Conclusions: This study shows that GBM-based propensity score weighting using balance-based stopping rules can remove observed pretreatment difference with multiple treatments. It also offers preliminary evidence that standalone urine drug screening is associated with reduced substance use and substance problems. Further work should explore whether standalone urine drug screening would be an effective treatment modality for substance using adolescents, and the conditions necessary for it to achieve its potential effects.