Choosing the Number of Clusters in Monothetic Clustering

• Activity Number: 552
• Type: Contributed
• Date/Time: Wednesday, August 12, 2015 : 10:30 AM to 12:20 PM
• Sponsor: Section on Statistical Learning and Data Mining
• Author(s): Mark C. Greenwood* and Tan V. Tran
• Companies: Montana State University and Montana State University
• Keywords: monothetic cluster analysis ; non-parametric ; cross-validation ; number of clusters

Abstract

Monothetic clustering is a divisive clustering method based on recursive bipartitions of the data set determined by choosing splitting rules from any of the variables to conditionally optimally partition the multivariate responses. Like in other clustering methods, the choice of the number of clusters is important in this method. Connections between monothetic clustering and decision trees motivate the consideration of pruning methods as aids in selecting the number of clusters. We apply different cross-validation techniques to find the number of clusters that optimize prediction error and compare that approach to permutation-based hypothesis tests at each bi-splitting step, retaining splits with "small" p-values. A simulation study is performed to evaluate the performance of the new methods and compare to some other existing techniques.