All Times EDT
Keywords: heterogeneous treatment effects, machine learning, causal inference, healthcare, healthcare insurance
Medical claims data are regularly used to evaluate the impact of interventions aimed at improving population health, but it is difficult to measure the true impact of interventions using observational data, such as claims data, due to confounding or targeted selection, especially when effects are small. Non-normality, non-linear relationships, and interactions add additional complexity. In order to make causal inferences we traditionally use propensity score matching to balance treatment and comparison groups, followed by a linear model to measure the impact. Because of the afore-mentioned complexities, linear models may not provide the best performance. We evaluated the performance of “heterogeneous treatment effects” methods for estimating average and conditional treatment effects. Two of the most renowned methods (the BART and causal forests (CF)) were compared to a linear model with propensity score matching. Our analysis is based on a simulation study that mimics the complexity of real claims data according to the following data generation process: Outcome = Prognostic function + Treatment effect function The prognostic function describes the relationship between covariates and the outcome. In presence of targeted selection (i.e. when assignment to the treatment group is influenced by the expected outcome), the propensity function depends on the prognostic function. In practice, the prognostic function was generated using real data, while the propensity function was generated artificially to mimic targeted selection. The treatment effect function was generated artificially using a linear relationship and interactions to mimic heterogeneity. Models were compared using the RMSE and coverage. When correctly specified, the linear model performed well, but it was outperformed by CF and BART under more complex scenarios. This suggests that using CF or BART could provide better estimates of intervention impact when using data from claims.