Flexible parametric bootstrap for testing homogeneity against clustering and assessing the number of clusters

DOI10.1007/S11222-015-9566-5zbMATH Open1331.62308arXiv1502.02574OpenAlexW1783400983WikidataQ59409829 ScholiaQ59409829MaRDI QIDQ5963783FDOQ5963783

Authors: Christian Hennig, Chien-Ju Lin 

Publication date: 23 February 2016

Published in: Statistics and Computing (Search for Journal in Brave)

Abstract: There are two notoriously hard problems in cluster analysis, estimating the number of clusters, and checking whether the population to be clustered is not actually homogeneous. Given a dataset, a clustering method and a cluster validation index, this paper proposes to set up null models that capture structural features of the data that cannot be interpreted as indicating clustering. Artificial datasets are sampled from the null model with parameters estimated from the original dataset. This can be used for testing the null hypothesis of a homogeneous population against a clustering alternative. It can also be used to calibrate the validation index for estimating the number of clusters, by taking into account the expected distribution of the index under the null model for any given number of clusters. The approach is illustrated by three examples, involving various different clustering techniques (partitioning around medoids, hierarchical methods, a Gaussian mixture model), validation indexes (average silhouette width, prediction strength and BIC), and issues such as mixed type data, temporal and spatial autocorrelation.

Full work available at URL: https://arxiv.org/abs/1502.02574

Recommendations

• Significance testing in clustering
• Identifying genuine clusters in a classification
• Identifying genuine clusters in a classification
• How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis
• Comparing clusterings and numbers of clusters by aggregation of calibrated clustering validity indexes

zbMATH Keywords

Markov chainmixture modelmixed-type dataspatial autocorrelationcluster validationdistance-based clusteringpresence-absence data

Mathematics Subject Classification ID

Parametric hypothesis testing (62F03) Bootstrap, jackknife and other resampling methods (62F40) Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Cites Work

• Finding Groups in Data
• Title not available (Why is that?)
• Model-Based Clustering, Discriminant Analysis, and Density Estimation
• Probabilistic models in cluster analysis
• Consistent estimation of the order of mixture models.
• How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis
• Finding the Number of Clusters in a Dataset
• Title not available (Why is that?)
• Title not available (Why is that?)
• Multidimensional scaling.
• Model-Based Clustering, Classification, and Density Estimation Using mclust in R
• Distance-based parametric bootstrap tests for clustering of species ranges

Cited In (9)

• Distance-based parametric bootstrap tests for clustering of species ranges
• Bootstrapping for Significance of Compact Clusters in Multidimensional Datasets
• Clustering with the average silhouette width
• Spatial variability clustering for spatially dependent functional data
• Distance Metrics and Clustering Methods for Mixed‐type Data
• E-ReMI: extended maximal interaction two-mode clustering
• An adequacy approach for deciding the number of clusters for OTRIMLE robust Gaussian mixture‐based clustering
• Testing independence between two nonhomogeneous point processes in time
• REMAXINT: a two-mode clustering-based method for statistical inference on two-way interaction

Uses Software

• mclust

This page was built for publication: Flexible parametric bootstrap for testing homogeneity against clustering and assessing the number of clusters

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5963783)