Cluster Validation by Measurement of Clustering Characteristics Relevant to the User

• Activity Number: 46 - Recent Advances in Cluster Analysis and Cluster Validation
• Type: Invited
• Date/Time: Sunday, July 29, 2018 : 4:00 PM to 5:50 PM
• Sponsor: Section on Statistical Learning and Data Science
• Author(s): Christian Hennig*
• Companies: University College London
• Keywords: Number of clusters; separation; homogeneity; density mode; random clustering

Abstract

There are many cluster analysis methods that can produce quite different clusterings on the same dataset. Cluster validation is about the evaluation of the quality of a clustering; "relative cluster validation" is about using such criteria to compare clusterings. This can be used to select one of a set of clusterings from different methods, or from the same method ran with different parameters such as different numbers of clusters. There are many cluster validation indexes in the literature. Most of them attempt to measure the overall quality of a clustering by a single number, but this can be inappropriate. There are various different characteristics of a clustering that can be relevant in practice, depending on the aim of clustering, such as low within-cluster distances and high between-cluster separation. In this paper, a number of validation criteria will be introduced that refer to different desirable characteristics of a clustering, and that characterise a clustering in a multidimensional way. In specific applications the user may be interested in some of these criteria rather than others.