Influential features PCA for high dimensional clustering

DOI10.1214/15-AOS1423MaRDI QIDQ510669zbMATH OpenOpenAlexFDO

Authors Wanjie Wang, Jiashun Jin

Publication date 13 February 2017

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1407.5241

zbMATH Keywords

empirical nullfeature selectiongene microarrayHamming distancephase transitionpost-selection spectral clusteringsparsity

Mathematics Subject Classification ID

Asymptotic distribution theory in statistics (62E20) Nonparametric hypothesis testing (62G10) Statistics of extreme values; tail inference (62G32) Factor analysis and principal components; correspondence analysis (62H25) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Applications of statistics to biology and medical sciences; meta analysis (62P10)

Abstract: We consider a clustering problem where we observe feature vectors $X_{i}in{R}^p$, $i=1,2,ldots,n$, from $K$ possible classes. The class labels are unknown and the main interest is to estimate them. We are primarily interested in the modern regime of $pggn$, where classical clustering methods face challenges. We propose Influential Features PCA (IF-PCA) as a new clustering procedure. In IF-PCA, we select a small fraction of features with the largest Kolmogorov-Smirnov (KS) scores, where the threshold is chosen by adapting the recent notion of Higher Criticism, obtain the first $(K-1)$ left singular vectors of the post-selection normalized data matrix, and then estimate the labels by applying the classical k-means to these singular vectors. It can be seen that IF-PCA is a tuning free clustering method. We apply IF-PCA to $10$ gene microarray data sets. The method has competitive performance in clustering. Especially, in three of the data sets, the error rates of IF-PCA are only $29\mathrm{\%}$ or less of the error rates by other methods. We have also rediscovered a phenomenon on empirical null by cite{Efron} on microarray data. With delicate analysis, especially post-selection eigen-analysis, we derive tight probability bounds on the Kolmogorov-Smirnov statistics and show that IF-PCA yields clustering consistency in a broad context. The clustering problem is connected to the problems of sparse PCA and low-rank matrix recovery, but it is different in important ways. We reveal an interesting phase transition phenomenon associated with these problems and identify the range of interest for each.

Review text

Starting from the problem of clustering using gene microarray data, the paper approaches the situation when the feature vectors come from different classes the labels of which are unknown. The authors propose as solution to this problem the influential features PCA (IF-PCA) technique as a new spectral clustering method, along with the Kolmogorov-Smirnov (K-S) score. Since the performance of IF-PCA depends on the choice of the corresponding threshold, the Higher Criticism (H-C) technique is used accordingly. The model is applied to ten different microarray medical datasets (brain, breast cancer, leukemia, etc.) and compared with other clustering methods, and the method is proved to be efficient.

Recommendations

• Phase transitions for high dimensional clustering and related problems
• Clustering High-Dimensional Data via Feature Selection
• Subspace clustering of high-dimensional data: a predictive approach
• A framework for feature selection in clustering
• Clustering and feature selection using sparse principal component analysis

Cited in

(29)

• Higher criticism for discriminating word-frequency tables and authorship attribution
• Feature screening in large scale cluster analysis
• Phase transitions for high dimensional clustering and related problems
• Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics
• Hierarchical clustering with optimal transport
• An \({\ell_p}\) theory of PCA and spectral clustering
• Testing equivalence of clustering
• scientific article; zbMATH DE number 7370563 (Why is no real title available?)
• Diagonally Dominant Principal Component Analysis
• Optimal sparse singular value decomposition for high-dimensional high-order data
• Spiked sample covariance matrices with possibly multiple bulk components
• Clustering High-Dimensional Data via Feature Selection
• Eigen Selection in Spectral Clustering: A Theory-Guided Practice
• Estimation of the Number of Spiked Eigenvalues in a Covariance Matrix by Bulk Eigenvalue Matching Analysis
• A Normality Test for High-dimensional Data Based on the Nearest Neighbor Approach
• Strategic oscillation for the balanced minimum sum-of-squares clustering problem
• Coordinatewise Gaussianization: Theories and Applications
• A Bayesian non-parametric approach for automatic clustering with feature weighting
• Feature screening for clustering analysis of count data with an application to single-cell RNA-sequencing
• Degree-Heterogeneous Latent Class Analysis for High-Dimensional Discrete Data
• Region detection and image clustering via sparse Kronecker product decomposition
• Multivariate normality test based on Shannon's entropy
• An efficient and distribution-free symmetry test for high-dimensional data based on energy statistics and random projections
• Limitations of clustering with PCA and correlated noise
• Uniform error bound for PCA matrix denoising
• Sharp-SSL: Selective High-Dimensional Axis-Aligned Random Projections for Semi-Supervised Learning
• Network-adjusted covariates for community detection
• Non-splitting Neyman-Pearson classifiers
• A Comparison of DeepSeek and other LLMs

This page was built for publication: Influential features PCA for high dimensional clustering

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q510669)