Bias of the Out-of-Bag (OOB) Error for Random Forests

• Activity Number: 179
• Type: Contributed
• Date/Time: Monday, August 1, 2011 : 10:30 AM to 12:20 PM
• Sponsor: ENAR
• Author(s): Matthew Mitchell*+
• Companies: Metabolon, Inc.
• Keywords: random forest ; out of bag error ; metabolomics

Abstract

Random Forest is an excellent classification tool, especially in the -omics sciences such as metabolomics where the number of variables is much greater than the number of subjects, i.e., "n < < p." However, the choices for the arguments for the random forest implementation are very important. With the default arguments, the out-of-bag (OOB) error overestimates the true error, i.e., the random forest actually performs better than indicated by the OOB error. This bias is greatly reduced by sampling without replacement and choosing the same number of samples from each group. However, even after these adjustments, there is a low amount of bias. The remaining bias occurs because when there are trees with equal predictive ability the one that performs better on the in-bag samples will perform worse on the out-of-bag samples. Cross-validation can be performed to reduce the remaining bias.