|
Abstract:
|
Extreme values in Amazon supply chain data result in low-powered experiments and high uncertainty when estimating mean treatment effects. Even when extreme values are not inappropriately trimmed or assumed to conform to central limit theorem behavior, independence is often assumed across treatment and control groups, and independent extreme value distributions are used to fit treatment and control tails. We argue that independent extreme value tails are overly conservative for AB-testing applications with heavy tails and small treatment effects, as the unobserved counterfactuals are likely close to their observed experimental value. These are appropriately modeled by extreme value distributions with a shared tail index across treatment and control. This implies that the distribution of differences between treatment and control is light-tailed, and permits mean treatment effect estimation even when the treatment and control population means do not exist. We illustrate gains using both simulated and real data.
|
