An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization
From MaRDI portal
(Redirected from Publication:2188245)
Abstract: We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM by appealing to the rich Riemannian geometry of positive definite matrices, using which we cast Gmm parameter estimation as a Riemannian optimization problem. Surprisingly, such an out-of-the-box Riemannian formulation completely fails and proves much inferior to EM. This motivates us to take a closer look at the problem geometry, and derive a better formulation that is much more amenable to Riemannian optimization. We then develop (Riemannian) batch and stochastic gradient algorithms that outperform EM, often substantially. We provide a non-asymptotic convergence analysis for our stochastic method, which is also the first (to our knowledge) such global analysis for Riemannian stochastic gradient. Numerous empirical results are included to demonstrate the effectiveness of our methods.
Recommendations
- Riemannian online algorithms for estimating mixture model parameters
- Recent Advances in Stochastic Riemannian Optimization
- Statistical convergence of the EM algorithm on Gaussian mixture models
- Maximum likelihood estimation of Gaussian mixture models without matrix operations
- Improved convergence guarantees for learning Gaussian mixture models by EM and gradient EM
Cites work
- scientific article; zbMATH DE number 6381735 (Why is no real title available?)
- scientific article; zbMATH DE number 5131267 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 681023 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- A survey and comparison of contemporary algorithms for computing the matrix geometric mean
- Conic geometric optimization on the manifold of positive definite matrices
- Finite mixture models
- Geodesic Convexity and Covariance Estimation
- Global rates of convergence for nonconvex optimization on manifolds
- Hessian Riemannian Gradient Flows in Convex Programming
- Learning mixtures of Gaussians in high dimensions
- Low-rank matrix completion by Riemannian optimization
- Low-rank optimization on the cone of positive semidefinite matrices
- Machine learning. A probabilistic perspective
- Manopt, a Matlab toolbox for optimization on manifolds
- Mixture Densities, Maximum Likelihood and the EM Algorithm
- Optimization methods on Riemannian manifolds and their application to shape space
- Riemannian metrics on positive definite matrices related to means
- Riemannian metrics on positive definite matrices related to means. II
- Statistical guarantees for the EM algorithm: from population to sample-based analysis
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
- Stochastic Gradient Descent on Riemannian Manifolds
- The elements of statistical learning. Data mining, inference, and prediction
- Theoretical statistics. Topics for a core course
Cited in
(17)- GAT–GMM: Generative Adversarial Training for Gaussian Mixture Models
- Learning with symmetric positive definite matrices via generalized Bures-Wasserstein geometry
- 3D human pose tracking priors using geodesic mixture models
- Second-order optimization over the multivariate Gaussian distribution
- Accelerated Optimization in the PDE Framework: Formulations for the Manifold of Diffeomorphisms
- Fenchel conjugate via Busemann function on Hadamard manifolds
- Parsimonious Gaussian mixture models with piecewise-constant eigenvalue profiles
- TLS-EM algorithm of mixture density models for exponential families
- A trust region method for solving multicriteria optimization problems on Riemannian manifolds
- A Riemannian Newton trust-region method for fitting Gaussian mixture models
- Riemannian online algorithms for estimating mixture model parameters
- Riemannian optimization for variance estimation in linear mixed models
- Large-sample properties of unsupervised estimation of the linear discriminant using projection pursuit
- Recent Advances in Stochastic Riemannian Optimization
- Orthogonal Stiefel manifold optimization for eigen-decomposed covariance parameter estimation in mixture models
- Nonconvex weak sharp minima on Riemannian manifolds
- Practical gradient and conjugate gradient methods on flag manifolds
This page was built for publication: An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2188245)
