Statistical Significance for Hierarchical Clustering
From MaRDI portal
Publication:6079976
DOI10.1111/BIOM.12647zbMATH Open1522.62165arXiv1411.5259OpenAlexW317995076WikidataQ47151800 ScholiaQ47151800MaRDI QIDQ6079976FDOQ6079976
Authors: Patrick K. Kimes, Yufeng Liu, David Neil Hayes, J. S. Marron 
Publication date: 30 October 2023
Published in: Biometrics (Search for Journal in Brave)
Abstract: Cluster analysis has proved to be an invaluable tool for the exploratory and unsupervised analysis of high dimensional datasets. Among methods for clustering, hierarchical approaches have enjoyed substantial popularity in genomics and other fields for their ability to simultaneously uncover multiple layers of clustering structure. A critical and challenging question in cluster analysis is whether the identified clusters represent important underlying structure or are artifacts of natural sampling variation. Few approaches have been proposed for addressing this problem in the context of hierarchical clustering, for which the problem is further complicated by the natural tree structure of the partition, and the multiplicity of tests required to parse the layers of nested clusters. In this paper, we propose a Monte Carlo based approach for testing statistical significance in hierarchical clustering which addresses these issues. The approach is implemented as a sequential testing procedure guaranteeing control of the family-wise error rate. Theoretical justification is provided for our approach, and its power to detect true clustering structure is illustrated through several simulation studies and applications to two cancer gene expression datasets.
Full work available at URL: https://arxiv.org/abs/1411.5259
Recommendations
- Statistical Significance of Clustering for High-Dimension, Low–Sample Size Data
- Significance testing in clustering
- A permutation test for determining significance of clusters with applications to spatial and gene expression data
- Spatial clustering of array CGH features in combination with hierarchical multiple testing
- A new algorithm for hybrid hierarchical clustering with visualization and the bootstrap
Cites Work
- Model-Based Clustering, Discriminant Analysis, and Density Estimation
- Finite mixture models
- Hierarchical testing of variable importance
- Eigenvalues of large sample covariance matrices of spiked population models
- Approximately unbiased tests of regions using multistep-multiscale bootstrap resampling
- Bootstrap confidence levels for phylogenetic trees
- Geometric Representation of High Dimension, Low Sample Size Data
- Statistical Significance of Clustering for High-Dimension, Low–Sample Size Data
- Bootstrapping for Significance of Compact Clusters in Multidimensional Datasets
- Asymptotics of hierarchical clustering for growing dimension
Cited In (14)
- Statistical Significance of Clustering with Multidimensional Scaling
- Post-clustering difference testing: valid inference and practical considerations with applications to ecological and biological data
- Coherence-based time series clustering for statistical inference and visualization of brain connectivity
- The next‐generation K‐means algorithm
- Selective Inference for Hierarchical Clustering
- A permutation test for determining significance of clusters with applications to spatial and gene expression data
- On some significance tests in cluster analysis
- Asymptotic properties of hierarchical clustering in high-dimensional settings
- U-Statistical Inference for Hierarchical Clustering
- A new algorithm for hybrid hierarchical clustering with visualization and the bootstrap
- Clustering by principal component analysis with Gaussian kernel in high-dimension, low-sample-size settings
- Statistical Significance of Clustering for High-Dimension, Low–Sample Size Data
- Nonparametric cluster significance testing with reference to a unimodal null distribution
- Towards U-statistics clustering inference for multiple groups
This page was built for publication: Statistical Significance for Hierarchical Clustering
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6079976)
