A Compressed PCA Subspace Method for Anomaly Detection in High-Dimensional Data

• Activity Number: 357
• Type: Contributed
• Date/Time: Tuesday, August 3, 2010 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Qi Ding*+ and Eric Kolaczyk
• Companies: Boston University and Boston University
• Keywords: random projection ; PCA ; anomaly detection

Abstract

Random projection is a widely used method of dimension reduction. Its combination with standard techniques of regression and classification has been explored recently. Here we examine its use with principal component analysis (PCA) based subspace detection methods. Specifically, we show that, under appropriate conditions, with high probability the magnitude of the residuals of a PCA analysis of randomly projected data behaves nearly the same as the PCA residuals of the original data. Our results indicate the feasibility of applying subspace-based anomaly detection algorithms to randomly projected data, when the nature of the data covariance is sparse. We illustrate in the context of computer network traffic anomaly detection.