Optimal estimation of high-dimensional Gaussian location mixtures

DOI10.1214/22-AOS2207MaRDI QIDQ6046303zbMATH OpenFDO

Authors Natalie Doss, Yihong Wu, Pengkun Yang, Harrison H. Zhou

Publication date 10 May 2023

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2002.05818

zbMATH Keywords

finite mixture modelGaussian mixturehigh-dimensional density estimationlow-rank tensormethod of momentsmetric entropyminimax optimality

Mathematics Subject Classification ID

Minimax procedures in statistical decision theory (62C20) Nonparametric estimation (62G05) Density estimation (62G07)

Abstract: This paper studies the optimal rate of estimation in a finite Gaussian location mixture model in high dimensions without separation conditions. We assume that the number of components $k$ is bounded and that the centers lie in a ball of bounded radius, while allowing the dimension $d$ to be as large as the sample size $n$. Extending the one-dimensional result of Heinrich and Kahn cite{HK2015}, we show that the minimax rate of estimating the mixing distribution in Wasserstein distance is $Theta((d/n{)}^{1/4}+n^{-1/(4k-2)})$, achieved by an estimator computable in time $O(n{d}^2+n^{5/4})$. Furthermore, we show that the mixture density can be estimated at the optimal parametric rate $Theta(sqrtd/n)$ in Hellinger distance and provide a computationally efficient algorithm to achieve this rate in the special case of $k=2$. Both the theoretical and methodological development rely on a careful application of the method of moments. Central to our results is the observation that the information geometry of finite Gaussian mixtures is characterized by the moment tensors of the mixing distribution, whose low-rank structure can be exploited to obtain a sharp local entropy bound.

Recommendations

• Minimax bounds for estimating multivariate Gaussian location mixtures
• Regularized parameter estimation in high-dimensional Gaussian mixture models
• Optimal estimation and computational limit of low-rank Gaussian mixtures
• Optimal estimation of Gaussian mixtures via denoised method of moments
• Optimal Bayesian estimation of Gaussian mixtures with growing number of components
• Maximum likelihood estimation of heterogeneous mixtures of Gaussian and uniform distributions
• General minimax estimators on the location matrix of a mixture of normals
• Parameter estimation in high dimensional Gaussian distributions
• On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising
• Learning mixtures of Gaussians in high dimensions

Cites work

• A non asymptotic penalized criterion for Gaussian mixture model selection
• A spectral algorithm for learning mixture models
• Adaptive Bayesian multivariate density estimation with Dirichlet mixtures
• Approximation dans les espaces m�triques et th�orie de l'estimation
• Asymptotic behaviour of the posterior distribution in overfitted mixture models
• Asymptotics for likelihood ratio tests under loss of identifiability
• Bayesian methods for the shape invariant model
• Consistent Order Estimation and Minimal Penalties
• Convergence of estimates under dimensionality restrictions
• Convergence of latent mixing measures in finite and infinite mixture models
• Convergence rates of parameter estimation for some weakly identifiable finite mixtures
• Convergence rates of posterior distributions.
• Convex Optimization, Shape Constraints, Compound Decisions, and Empirical Bayes Rules
• CVXPY: a Python-embedded modeling language for convex optimization
• Dissipation of Information in Channels With Input Constraints
• Efficiently learning mixtures of two Gaussians
• Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities.
• Estimation of analytic functions
• Generalized maximum likelihood estimation of normal mixture densities
• scientific article; zbMATH DE number 5654889 (Why is no real title available?)
• scientific article; zbMATH DE number 1909499 (Why is no real title available?)
• scientific article; zbMATH DE number 6791314 (Why is no real title available?)
• scientific article; zbMATH DE number 3273551 (Why is no real title available?)
• scientific article; zbMATH DE number 3107192 (Why is no real title available?)
• Inconsistency of Pitman-Yor process mixtures for the number of components
• Introduction to nonparametric estimation
• Large Sample Properties of Generalized Method of Moments Estimators
• Learning mixtures of separated nonspherical Gaussians
• Learning Theory
• Learning Theory
• Minimax bounds for estimation of normal mixtures
• Mixture models, robustness, and sum of squares proofs
• Moment matrices: Applications in mixtures
• Nuclear norm of higher-order tensors
• On estimating a density using Hellinger distance and some other strange facts
• On posterior contraction of parameters and interpretability in Bayesian mixture modeling
• On strong identifiability and convergence rates of parameter estimation in finite mixtures
• On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising
• Optimal estimation of Gaussian mixtures via denoised method of moments
• Optimal rate of convergence for finite mixture models
• PAC Learning Axis-Aligned Mixtures of Gaussians with No Separation Assumption
• Posterior consistency of Dirichlet mixtures in density estimation
• Posterior convergence rates of Dirichlet mixtures at smooth densities
• Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in \(O(\sqrt{n})\) iterations
• Rates of convergence for the Gaussian mixture sieve.
• Rates of convergence of posterior distributions.
• Smallest singular value of a random rectangular matrix
• Strong equivalence between metrics of Wasserstein type
• Strong identifiability and optimal minimax rates for finite mixture estimation
• Sub-Gaussian estimators of the mean of a random vector
• Symmetric Tensors and Symmetric Tensor Rank
• Tensor Decompositions and Applications
• Testing the order of a finite mixture
• The best rank-one approximation ratio of a tensor space
• The estimation of the order of a mixture model
• The local geometry of finite mixtures
• The Rotation of Eigenvectors by a Perturbation. III
• The statistical work of Lucien Le Cam.
• Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics
• Tight bounds for learning a mixture of two Gaussians (extended abstract)
• Weak convergence and empirical processes. With applications to statistics
• Über homogene Polynome in ($L^{2}$)

Cited in

(15)

• Implicit models of Gaussian mixture densities and locally optimum detectors
• Sharp optimal recovery in the two component Gaussian mixture model
• Computing tail areas for a high-dimensional Gaussian mixture
• Optimal estimation and computational limit of low-rank Gaussian mixtures
• A generalized Isserlis theorem for location mixtures of Gaussian random vectors
• Empirical optimal transport under estimated costs: distributional limits and statistical applications
• Gaussian mixtures closest to a given measure via optimal transport
• Poisson empirical Bayes estimation: when does g-modeling beat f-modeling in theory (and in practice)?
• Multivariate, heteroscedastic empirical Bayes via nonparametric maximum likelihood
• Weighted support vector machine for extremely imbalanced data
• Sharp-SSL: Selective High-Dimensional Axis-Aligned Random Projections for Semi-Supervised Learning
• Tensor eigenvectors for projection pursuit
• Quantization of probability distributions via divide-and-conquer: convergence and error propagation under distributional arithmetic operations
• Two statistical problems for multivariate mixture distributions
• Convergence and optimality of the EM algorithm under multi-component Gaussian mixture models

This page was built for publication: Optimal estimation of high-dimensional Gaussian location mixtures

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6046303)