Combining Random Forest with Targeted Maximum Likelihood Estimation for Variable Importance Analysis

• Activity Number: 340
• Type: Contributed
• Date/Time: Tuesday, July 31, 2012 : 10:30 AM to 12:20 PM
• Sponsor: Biometrics Section
• Author(s): Hui Wang*+ and Mark van der Laan
• Companies: Stanford University and University of California at Berkeley School of Public Health
• Keywords: TMLE ; semiparametric ; random forest ; cross validation ; overfitting

Abstract

Targeted maximum likelihood estimation (TMLE) is a general estimation methodology, particularly useful for parameter estimation in semiparametric models. Algorithmically, TMLE is a two-stage procedure. First, an initial estimator of the overall density is obtained. Second, the initial estimator is updated in a direction targeted towards the parameter of interest, often involving estimation of nuisance portion of the likelihood. TMLE is sensitive to overfitting in the initial estimator. This problem is particularly true when Random Forest is used in the first stage. To address this problem, we propose a modified TMLE --- cross validated TMLE to combine with Random Forest. The cross validated TMLE uses cross validated predictions in the first stage of TMLE, instead of empirical predictions. This modification substantially improves the resulting estimator, correcting the overfitting problem in the initial estimator. Simulations and real data analyses are both conducted to demonstrate cross validated TMLE.