On the meaning of Dunn's partition coefficient for fuzzy clusters (Q1104672)

A constantly recurring problem in cluster analysis is that of evaluating the number of clusters that are present in a data set. The development of fuzzy clustering has brought no definite answer to this question, although some new means have been provided offering new ways to tackle this problem. One of the promising approaches was Dunn's partition coefficient \(F_ k(U)\) [see \textit{J. C. Dunn}, J. Cybernet. 4, 1-15 and 95-104 (1974)] that has been shown to vary between 1, for hard clusters, and 1/k, for completely fuzzy sets of objects: hence the idea that \(F_ k(U)\) expresses a measure of how far a given fuzzy partition is from a hard one. Assuming moreover that an optimal partition in an optimal number of clusters will have a `harder' look than any other partition, \(F_ k(U)\) could be considered as a cluster validity index. That this procedure is unjustified is shown by the study of the (D,F) diagram in which, by analogy to the partition coefficient F, a fuzziness coefficient D is defined. An important result of this study is that a higher \(F_ k(U)\) value does not always correspond to a better allocation than a partition with a lower value: this observation contradicts the role of the partition coefficient as cluster validity measurement. These results are amply confirmed by various artificial and classical numerical examples.