Truncated Outlier Filtering
*Dr. Peter J. Costa, Hologic Incorporated 

Keywords: Order statistics, filter, outliers

The statistical analysis of data can be distorted by measurements which do not reflect the qualitative nature of the population from which they are drawn. Such spurious measurements are called outliers. Conventional techniques identify those measurements whose distance from the mean exceed a selected quantile multiple of the standard deviation as outliers. Such approaches, however, can fail to associate measurements with large normalized distances as outliers. The truncated outlier method filters the minimum and maximum of the population before computing the exclusion criterion. This mitigates the influence of abnormal measurements on the normalized distance and yields a more stable criterion for outlier determination. Moreover, the method generalizes to higher dimensions (n = 2). Simulated one-dimensional and multi-dimensional data sets are analyzed. A discussion of the results is also presented.