A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics
From MaRDI portal
Publication:2174980
Abstract: We propose a new method of estimation in topic models, that is not a variation on the existing simplex finding algorithms, and that estimates the number of topics K from the observed data. We derive new finite sample minimax lower bounds for the estimation of A, as well as new upper bounds for our proposed estimator. We describe the scenarios where our estimator is minimax adaptive. Our finite sample analysis is valid for any number of documents (n), individual document length (N_i), dictionary size (p) and number of topics (K), and both p and K are allowed to increase with n, a situation not handled well by previous analyses. We complement our theoretical results with a detailed simulation study. We illustrate that the new algorithm is faster and more accurate than the current ones, although we start out with a computational and theoretical disadvantage of not knowing the correct number of topics K, while we provide the competing methods with the correct value in our simulations.
Recommendations
- Sparse Topic Modeling: Computational Efficiency, Near-Optimal Algorithms, and Statistical Inference
- Optimal estimation of sparse topic models
- Learning Topic Models: Identifiability and Finite-Sample Analysis
- Convergence of the algorithm of additive regularization of topic models
- Convergence rates of latent topic models under relaxed identifiability conditions
- Inference for the Number of Topics in the Latent Dirichlet Allocation Model via Bayesian Mixture Modeling
- Topic Models and Nonnegative Matrix Factorization
- On Confidence-Constrained Rank Recovery in Topic Models
Cites work
- 10.1162/jmlr.2003.3.4-5.993
- A correlated topic model of science
- A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics
- A spectral algorithm for latent Dirichlet allocation
- scientific article; zbMATH DE number 3998953 (Why is no real title available?)
- Latent semantic indexing: A probabilistic analysis
Cited in
(14)- Using SVD for Topic Modeling
- Likelihood estimation of sparse topic distributions in topic models and its applications to Wasserstein document distance calculations
- A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics
- Adaptive estimation in structured factor models with applications to overlapping clustering
- Optimal Bayesian estimation of Gaussian mixtures with growing number of components
- Detecting approximate replicate components of a high-dimensional random vector with latent structure
- Statistical Methods for Minimax Estimation in Linear Models with Unknown Design Over Finite Alphabets
- Optimal estimation of sparse topic models
- Sparse Topic Modeling: Computational Efficiency, Near-Optimal Algorithms, and Statistical Inference
- Assigning topics to documents by successive projections
- Special invited paper: the SCORE normalization, especially for heterogeneous network and text data
- Robust machine learning algorithms for text analysis
- Learning topic hierarchies by tree-directed latent variable models
- Estimation and inference for the Wasserstein distance between mixing measures in topic models
This page was built for publication: A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2174980)
