Random KNN Classification and Regression

• Activity Number: 343
• Type: Contributed
• Date/Time: Tuesday, August 5, 2014 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): E. James Harner*+ and Shengqiao Li and Donald Adjeroh
• Companies: West Virginia University and UPMC Health Plan and West Virginia University
• Keywords: Statistical Learning ; K-Nearest Neighbor ; High Dimensional Data ; Parallel Computing

Abstract

Random KNN (RKNN) is a generalization of traditional nearest-neighbor modeling. Random KNN consists of an ensemble of base k-nearest neighbor models, each constructed from a random subset of the input variables. Random KNN can be used to select important features using the RKNN-FS algorithm. Empirical results on microarray data sets with thousands of variables and relatively few samples show that RKNN-FS is an effective feature selection approach for high-dimensional data. RKNN is similar to Random Forests (RF) in terms of classification accuracy without feature selection. However, RKNN provides much better classification accuracy than RF when each method incorporates a feature-selection step. RKNN is significantly more stable and robust than Random Forests. Further, RKNN-FS is much faster than the Random Forests feature selection method (RF-FS), especially for large scale problems involving thousands of variables and/or multiple classes. Random KNN and feature selection algorithms are implemented in an R package rknn, which supports both classification and regression. We will show how to apply the Random KNN method via the rknn package to high-dimensional genomic data.