Outlier Detection for High-Dimensional Data

• Activity Number: 476
• Type: Invited
• Date/Time: Wednesday, August 6, 2014 : 10:30 AM to 12:20 PM
• Sponsor: IMS
• Author(s): Jeongyoun Ahn*+ and Myung Hee Lee and Jung Ae Lee
• Companies: University of Georgia and Colorado State University and Washington University in St. Louis
• Keywords: High Dimension Asymptotics ; Masking Effect ; Parametric Boostrap ; QQ Plot

Abstract

Despite the popularity of high dimension, low sample size data analysis, little attention has been paid to outlier detection. A main challenge is that there are not enough observations to measure the remoteness of a potential outlier. We consider three distance measures and study their high-dimensional properties regarding their abilities to identify multiple outliers when the dimension is much larger than the sample size. Using these distances, we propose an effective outlier detection algorithm that utilizes parametric bootstrap to obtain null distance values under the assumption of no outlier. A graphical diagnostic method that compares the observed one-vs-rest distances with null distances is also proposed. Both simulated and real data are used to demonstrate the performance of the proposed method in various population settings. .