Semi-supervised cross-entropy clustering with information bottleneck constraint

DOI10.1016/J.INS.2017.07.016MaRDI QIDQ780993zbMATH OpenOpenAlexFDO

Authors Marek Śmieja, Bernhard C. Geiger

Publication date 16 July 2020

Published in Information Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1705.01601

zbMATH Keywords

model-based clusteringcross-entropyinformation bottlenecksemi-supervised clusteringpartition-level side information

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30)

Abstract: In this paper, we propose a semi-supervised clustering method, CEC-IB, that models data with a set of Gaussian distributions and that retrieves clusters based on a partial labeling provided by the user (partition-level side information). By combining the ideas from cross-entropy clustering (CEC) with those from the information bottleneck method (IB), our method trades between three conflicting goals: the accuracy with which the data set is modeled, the simplicity of the model, and the consistency of the clustering with side information. Experiments demonstrate that CEC-IB has a performance comparable to Gaussian mixture models (GMM) in a classical semi-supervised scenario, but is faster, more robust to noisy labels, automatically determines the optimal number of clusters, and performs well when not all classes are present in the side information. Moreover, in contrast to other semi-supervised models, it can be successfully applied in discovering natural subgroups if the partition-level side information is derived from the top levels of a hierarchical clustering.

Recommendations

• Cross-entropy clustering
• Semi-supervised information-maximization clustering
• Fast semi-supervised evidential clustering
• scientific article; zbMATH DE number 1945220
• The information bottleneck and geometric clustering

Cites work

• scientific article; zbMATH DE number 5957257 (Why is no real title available?)
• A graph-based semi-supervised \(k\) nearest-neighbor method for nonlinear manifold distributed data classification
• Algorithm AS 136: A K-Means Clustering Algorithm
• Cross-entropy clustering
• Introduction to semi-supervised learning.
• Learning from partially supervised data using mixture models and belief functions
• Mixture Densities, Maximum Likelihood and the EM Algorithm
• Robust supervised classification with mixture models: learning from data with uncertain labels
• Semi-supervised concept factorization for document clustering
• Semi-supervised information-maximization clustering

Cited in

(1)

• Efficient mixture model for clustering of sparse high dimensional binary data

Describes a project that uses

Uses Software

• CEC
• AS 136
• Rmixmod

This page was built for publication: Semi-supervised cross-entropy clustering with information bottleneck constraint

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q780993)