Clustering with statistical error control

DOI10.1111/SJOS.12450zbMATH Open1473.62230arXiv1702.02643OpenAlexW3005549209MaRDI QIDQ5152167FDOQ5152167

Authors: Michael Vogt, Matthias Schmid 

Publication date: 17 September 2021

Published in: Scandinavian Journal of Statistics (Search for Journal in Brave)

Abstract: This paper presents a clustering approach that allows for rigorous statistical error control similar to a statistical test. We develop estimators for both the unknown number of clusters and the clusters themselves. The estimators depend on a tuning parameter alpha which is similar to the significance level of a statistical hypothesis test. By choosing alpha, one can control the probability of overestimating the true number of clusters, while the probability of underestimation is asymptotically negligible. In addition, the probability that the estimated clusters differ from the true ones is controlled. In the theoretical part of the paper, formal versions of these statements on statistical error control are derived in a standard model setting with convex clusters. A simulation study and two applications to temperature and gene expression microarray data complement the theoretical analysis.

Full work available at URL: https://arxiv.org/abs/1702.02643

Recommendations

• Statistical properties of convex clustering
• Estimating the number of clusters in a data set via the gap statistic
• Clustering noise-included data by controlling decision errors
• Estimating the number of clusters
• Clustering data with measurement errors

zbMATH Keywords

cluster analysis\(k\)-means clusteringnumber of clustersmultiple statistical testingstatistical error control

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Applications of statistics to biology and medical sciences; meta analysis (62P10)

Cited In (9)

• Clustering confidence sets
• Homogeneity Pursuit in Single Index Models based Panel Data Analysis
• Confidence set for group membership
• Clustering data with measurement errors
• Clustering noise-included data by controlling decision errors
• Estimating and clustering curves in the presence of heteroscedastic errors
• Impact of Contamination on Training and Test Error Rates in Statistical Clustering
• A statistical view of clustering performance through the theory of \(U\)-processes
• A new paradigm for high-dimensional data: distance-based semiparametric feature aggregation framework via between-subject attributes

This page was built for publication: Clustering with statistical error control

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5152167)