Online Program Home
My Program

Abstract Details

Activity Number: 480 - Model Testing and Prediction
Type: Contributed
Date/Time: Wednesday, August 1, 2018 : 8:30 AM to 10:20 AM
Sponsor: Business and Economic Statistics Section
Abstract #330716 Presentation
Title: Quantile Regression for AB Testing
Author(s): Luke Smith*
Companies: Amazon
Keywords: supply chain; quantile regression; causal inference; AB testing

Inventory Planning and Control Laboratory (IPC Lab) runs in-production supply chain experiments to permit Amazon supply chain researchers to investigate the impacts of new forecasting, buying, and placement ideas. In this talk we describe methodology for estimation of mean treatment effects. Causal inference in IPC Lab is challenging due to the high proportions of zeros, extreme right-tailed skewness, and treatment effects that are usually small, beneficial, and heterogeneous across the population. We investigate tail treatment effects using Bayesian quantile regression. We use a heavy-tailed density for the residuals, and leverage external information regarding reasonable magnitudes of population-level and individual-level treatment effects. Additionally, we argue that mean imputation of the counterfactual, as is commonly assumed in mean causal inference, poorly approximates the treatment effect in experiments with heavy-tailed data and small treatment effects. We compare our quantile regression model to mean regression and random forest, and show that Bayesian quantile regression leads to smaller standard errors and no degradation in out-of-sample predictive error.

Authors who are presenting talks have a * after their name.

Back to the full JSM 2018 program