Think Before You Cluster: Testing for Clusterability

• Activity Number: 46 - Recent Advances in Cluster Analysis and Cluster Validation
• Type: Invited
• Date/Time: Sunday, July 29, 2018 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Naomi Brownstein* and Margareta Ackerman and Andreas Adolfsson and Zachariah Neville
• Companies: Florida State University and Santa Clara University and Santa Clara University and Florida State University
• Keywords: Clusterability; clustering; dimension reduction; multimodality testing; software implementation; validation

Abstract

Clustering is a tool used throughout science to divide data into meaningful groups. Clusterability is a newer field quantifying the inherent cluster structure in a dataset. The goal of a clusterability test, applied before clustering, is to serve as a validation tool to alert researchers in the event that they have data lacking inherent cluster structure. For such data, deemed "unclusterable", cluster analysis should not be applied. We compare clusterability methods for their ability to identify data as containing -- or, critically, NOT containing -- evidence of multiple inherent clusters. Simulations evaluate type I error and power, as well as behavior for data with small clusters. Methods are applied to real datasets from a variety of different fields including biology, economics, and political science. Finally, we discuss the implementation of clusterability tests in standard software.