Deep mixtures of unigrams for uncovering topics in textual data

DOI10.1007/S11222-020-09989-9zbMATH Open1475.62063arXiv1902.06615OpenAlexW3135844644MaRDI QIDQ129168FDOQ129168

Cinzia Viroli, Laura Anderlucci

Publication date: 3 March 2021

Published in: Statistics and Computing (Search for Journal in Brave)

Abstract: Mixtures of Unigrams are one of the simplest and most efficient tools for clustering textual data, as they assume that documents related to the same topic have similar distributions of terms, naturally described by Multinomials. When the classification task is particularly challenging, such as when the document-term matrix is high-dimensional and extremely sparse, a more composite representation can provide better insight on the grouping structure. In this work, we developed a deep version of mixtures of Unigrams for the unsupervised classification of very short documents with a large number of terms, by allowing for models with further deeper latent layers; the proposal is derived in a Bayesian framework. The behaviour of the Deep Mixtures of Unigrams is empirically compared with that of other traditional and state-of-the-art methods, namely $k$-means with cosine distance, $k$-means with Euclidean distance on data transformed according to Semantic Analysis, Partition Around Medoids, Mixture of Gaussians on semantic-based transformed data, hierarchical clustering according to Ward's method with cosine dissimilarity, Latent Dirichlet Allocation, Mixtures of Unigrams estimated via the EM algorithm, Spectral Clustering and Affinity Propagation clustering. The performance is evaluated in terms of both correct classification rate and Adjusted Rand Index. Simulation studies and real data analysis prove that going deep in clustering such data highly improves the classification accuracy.

Full work available at URL: https://arxiv.org/abs/1902.06615

Recommendations

• Mixtures of Dirichlet-multinomial distributions for supervised and unsupervised classification of short text data
• Two-way Poisson mixture models for simultaneous document classification and word clustering
• Cluster-based sparse topical coding for topic mining and document clustering
• scientific article; zbMATH DE number 2086540
• scientific article; zbMATH DE number 1302160

zbMATH Keywords

clusteringdeep learningmixture modelstext data analysis

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Clustering by Passing Messages Between Data Points
• Finding Groups in Data
• Title not available (Why is that?)
• Model-Based Clustering, Discriminant Analysis, and Density Estimation
• Finite mixture models
• Least squares quantization in PCM
• Modelling high-dimensional data by mixtures of factor analyzers
• Dimensionally reduced model-based clustering through mixtures of factor mixture analyzers
• Heteroscedastic factor mixture analysis
• Deep Gaussian mixture models
• Text classification from labeled and unlabeled documents using EM
• 10.1162/jmlr.2003.3.4-5.993
• Ward's hierarchical agglomerative clustering method: which algorithms implement Ward's criterion?
• Adjusting for chance clustering comparison measures

Cited In (4)

• Study on text representation method based on deep learning and topic information
• Dealing with overdispersion in multivariate count data
• A mixture model approach to spectral clustering and application to textual data
• deepMOU

This page was built for publication: Deep mixtures of unigrams for uncovering topics in textual data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q129168)