Randomized Dual Rotations for High-Dimensional Outlier Detection

• Activity Number: 167 - Special Issues in Modeling
• Type: Contributed
• Date/Time: Monday, July 31, 2017 : 10:30 AM to 12:20 PM
• Sponsor: Korean International Statistical Society
• Author(s): Jeongyoun Ahn* and Hee Cheol Chung and Myung Hee Lee
• Companies: University of Georgia and University of Georgia and Weill Cornell Medical College
• Keywords: Dual space ; Generalized permutations ; Elliptical distribution

Abstract

Detecting outliers is a crucial step in any data analysis. A novel method for high dimensional outlier identification is proposed. We develop a new data perturbation method called randomized dual rotation with which one can simulate data sets that inherit the second moment properties of the original data but are free of outliers. We propose a two-stage procedure to detect outliers for high dimensional data. In the first stage, the whole data set is divided into two non-overlapping subsets: a set of non-outliers and a set of candidate outliers. Next, we test whether or not each candidate is substantially abnormal with a help of the empirical null distribution generated by dual rotations. The proposed approach is demonstrated with simulated and real data examples and shown to be competitive, especially with regard to specificity.