Learning Topic Models: Identifiability and Finite-Sample Analysis
From MaRDI portal
Abstract: Topic models provide a useful text-mining tool for learning, extracting, and discovering latent structures in large text corpora. Although a plethora of methods have been proposed for topic modeling, lacking in the literature is a formal theoretical investigation of the statistical identifiability and accuracy of latent topic estimation. In this paper, we propose a maximum likelihood estimator (MLE) of latent topics based on a specific integrated likelihood that is naturally connected to the concept, in computational geometry, of volume minimization. Our theory introduces a new set of geometric conditions for topic model identifiability, conditions that are weaker than conventional separability conditions, which typically rely on the existence of pure topic documents or of anchor words. Weaker conditions allow a wider and thus potentially more fruitful investigation. We conduct finite-sample error analysis for the proposed estimator and discuss connections between our results and those of previous investigations. We conclude with empirical studies employing both simulated and real datasets.
Recommendations
- Convergence rates of latent topic models under relaxed identifiability conditions
- Topic Models and Nonnegative Matrix Factorization
- Topic analysis using a finite mixture model.
- Inference for the Number of Topics in the Latent Dirichlet Allocation Model via Bayesian Mixture Modeling
- Probabilistic topic models for sequence data
- Optimal estimation of sparse topic models
- Sparse Topic Modeling: Computational Efficiency, Near-Optimal Algorithms, and Statistical Inference
Cites work
- 10.1162/jmlr.2003.3.4-5.993
- A spectral algorithm for latent Dirichlet allocation
- Blind Separation of Quasi-Stationary Sources: Exploiting Convex Geometry in Covariance Domain
- Convergence rates of latent topic models under relaxed identifiability conditions
- Estimating the endpoint of a distribution in the presence of additive observation errors
- Estimation of convex supports from noisy measurements
- Integrated likelihood methods for eliminating nuisance parameters. (With comments and a rejoinder).
- Latent semantic indexing: A probabilistic analysis
- Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition
- Nonnegative matrix factorization via archetypal analysis
- On the complexity of four polyhedral set containment problems
- Posterior contraction of the population polytope in finite admixture models
- Sparse Partially Collapsed MCMC for Parallel Inference in Topic Models
- Spectral analysis of data
- Tensor decompositions for learning latent variable models
- The stochastic EM algorithm: Estimation and asymptotic results
- Using mixture models for collaborative filtering
Cited in
(6)- Robust supervised topic models under label noise
- A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics
- Higher-order entrywise eigenvectors analysis of low-rank random matrices: bias correction, Edgeworth expansion and bootstrap
- Functional Mixed Membership Models
- Deep Discrete Encoders: Identifiable Deep Generative Models for Rich Data with Discrete Latent Layers
- Learning topic hierarchies by tree-directed latent variable models
This page was built for publication: Learning Topic Models: Identifiability and Finite-Sample Analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6185581)
