Random Forest Variable Selection Among Correlated Variables

• Activity Number: 643
• Type: Contributed
• Date/Time: Thursday, August 8, 2013 : 8:30 AM to 10:20 AM
• Sponsor: Biometrics Section
• Author(s): Joy Toyama*+ and Christina Kitchen
• Companies: and UCLA
• Keywords: random forests ; decision trees ; nonparametric

Abstract

Determining associations among clinical phenotypes for HIV therapeutics is a difficult problem due to the large number of potential predictors of the related immunophenotypic variables compared to the number of observations. Further exacerbating this problem is the fact that the predictor variables are highly correlated. Ensemble classifiers have been developed and used to handle such data. Random forests is a well-known nonparametric method which can illustrate complex interactions between the immunophenotpyic variables. In general, random forests present more stable results than tree classifiers alone and other ensemble methods. While this key characteristic holds true, it has been shown that variable importance measures can lead to biased results when there is high correlation among the potential predictors. When the goal is generating stable lists of "important variables" as opposed to prediction, allowing correlated variables to be selected is important. We propose a modification that includes more randomness to break this correlation and allow random forests to choose correlated variables more often and thus improve accuracy of the variance importance metrics.