Clustering of High-Dimensional Categorical Data

• Activity Number: 614
• Type: Contributed
• Date/Time: Wednesday, August 12, 2015 : 2:00 PM to 3:50 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Saeid Amiri* and Bertrand Clarke and Jennifer Clarke
• Companies: University of Nebraska - Lincoln and University of Nebraska - Lincoln and University of Nebraska - Lincoln
• Keywords: Clustering ; categorical data ; ensemble methods ; high dimensional data

Abstract

Here, we propose an ensemble approach to clustering categorical data. The proposed ensemble method is based on hierarchical clustering under average linkage. We give a rationale for why our procedure does well in low dimensions. This is supported by extensive computational comparisons with other methods using simulated and real data. Our method for low dimensional categorical data extends to high dimensional categorical data by using an extra level of ensembling. This minimizes the effect of the Curse of Dimensionality that tends to equalize the distances between any two points as dimension increases. A further extension of our ensembling method permits the vectors of categorical outcomes to have different dimensions.