Statistical guarantees for the EM algorithm: from population to sample-based analysis
From MaRDI portal
Abstract: We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in the limit of infinite data), followed by results that apply to updates based on a finite set of samples. First, we characterize the domain of attraction of any global maximizer of the population likelihood. This characterization is based on a novel view of the EM updates as a perturbed form of likelihood ascent, or in parallel, of the gradient EM updates as a perturbed form of standard gradient ascent. Leveraging this characterization, we then provide non-asymptotic guarantees on the EM and gradient EM algorithms when applied to a finite set of samples. We develop consequences of our general theory for three canonical examples of incomplete-data problems: mixture of Gaussians, mixture of regressions, and linear regression with covariates missing completely at random. In each case, our theory guarantees that with a suitable initialization, a relatively small number of EM (or gradient EM) steps will yield (with high probability) an estimate that is within statistical error of the MLE. We provide simulations to confirm this theoretically predicted behavior.
Recommendations
- Statistical convergence of the EM algorithm on Gaussian mixture models
- Convergence theorems for generalized alternating minimization procedures
- Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in \(O(\sqrt{n})\) iterations
- scientific article; zbMATH DE number 4032799
- On the global and componentwise rates of convergence of the EM algorithm
Cited in
(67)- Supermix: sparse regularization for mixtures
- Sketched learning for image denoising
- GAT–GMM: Generative Adversarial Training for Gaussian Mixture Models
- A new class of stochastic EM algorithms. Escaping local maxima and handling intractable sampling
- Robust high dimensional expectation maximization algorithm via trimmed hard thresholding
- Optimal estimation of Gaussian mixtures via denoised method of moments
- Geometry of EM and related iterative algorithms
- Analysis of a generalised expectation-maximisation algorithm for Gaussian mixture models: a control systems perspective
- Sequential estimation for mixture of regression models for heterogeneous population
- Oscillating neural circuits: Phase, amplitude, and the complex normal distribution
- A new algorithm for inference in HMM's with lower span complexity
- Loss modeling with the size-biased lognormal mixture and the entropy regularized EM algorithm
- scientific article; zbMATH DE number 7370581 (Why is no real title available?)
- Uniform consistency in nonparametric mixture models
- Moment Estimation for Nonparametric Mixture Models through Implicit Tensor Decomposition
- Statistical Inference with Local Optima
- Rate optimal estimation and confidence intervals for high-dimensional regression with missing covariates
- Improved convergence guarantees for learning Gaussian mixture models by EM and gradient EM
- scientific article; zbMATH DE number 7415097 (Why is no real title available?)
- A diffusion process perspective on posterior contraction rates for parameters
- Covariate regularized community detection in sparse graphs
- Finding low-rank solutions via nonconvex matrix factorization, efficiently and provably
- Computing Robust Statistics via an EM Algorithm
- Singularity, misspecification and the convergence rate of EM
- Hidden Integrality and Semirandom Robustness of SDP Relaxation for Sub-Gaussian Mixture Model
- Parameter recovery in two-component contamination mixtures: the \(L^2\) strategy
- A general frame for uncertainty propagation under multimodally distributed random variables
- Universal inference
- A Doubly Enhanced EM Algorithm for Model-Based Tensor Clustering
- Subgroup-effects models for the analysis of personal treatment effects
- Solution manifold and its statistical applications
- Tuning-free sparse clustering via alternating hard-thresholding
- Sharp global convergence guarantees for iterative nonconvex optimization with random data
- Exponential-Family Embedding With Application to Cell Developmental Trajectories for Single-Cell RNA-Seq Data
- On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising
- Optimal estimation and computational limit of low-rank Gaussian mixtures
- Statistical convergence of the EM algorithm on Gaussian mixture models
- Network inference from temporally dependent grouped observations
- STORE: sparse tensor response regression and neuroimaging analysis
- Estimating finite mixtures of ordinal graphical models
- Mixture conditional regression with ultrahigh dimensional text data for estimating extralegal factor effects
- Deep parameterizations of pairwise and triplet Markov models for unsupervised classification of sequential data
- Likelihood-based analysis in mixture global vars
- From inexact optimization to learning via gradient concentration
- Statistical analysis of Markov switching vector autoregression models with endogenous explanatory variables
- Fundamental limits of low-rank matrix estimation with diverging aspect ratios
- Nonparametric Finite Mixture of Gaussian Graphical Models
- Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems
- Improving the accuracy and internal consistency of regression-based clustering of high-dimensional datasets
- Community detection with dependent connectivity
- Simultaneous Clustering and Estimation of Heterogeneous Graphical Models
- The computational asymptotics of Gaussian variational inference and the Laplace approximation
- Reliable clustering of Bernoulli mixture models
- A flexible probabilistic framework for large-margin mixture of experts
- Statistical and computational guarantees for the Baum-Welch algorithm
- An alternative to EM for Gaussian mixture models: batch and stochastic Riemannian optimization
- Likelihood Maximization and Moment Matching in Low <scp>SNR</scp> Gaussian Mixture Models
- Large-sample properties of unsupervised estimation of the linear discriminant using projection pursuit
- Statistical Inference for High-Dimensional Vector Autoregression with Measurement Error
- On the curved exponential family in the stochastic approximation expectation maximization algorithm
- A tensor-EM method for large-scale latent class analysis with binary responses
- Semiparametric mixture regression with unspecified error distributions
- scientific article; zbMATH DE number 7370585 (Why is no real title available?)
- Homomorphic sensing of subspace arrangements
- Estimating a network from multiple noisy realizations
- Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in \(O(\sqrt{n})\) iterations
- Iterative algorithm for discrete structure recovery
This page was built for publication: Statistical guarantees for the EM algorithm: from population to sample-based analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q524451)
