On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising

DOI10.1214/19-AOS1817zbMATH Open1454.62120arXiv1712.02009OpenAlexW3029017346MaRDI QIDQ2196192FDOQ2196192

Adityanand Guntuboyina, Sujayam Saha

Publication date: 28 August 2020

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We study the Nonparametric Maximum Likelihood Estimator (NPMLE) for estimating Gaussian location mixture densities in $d$-dimensions from independent observations. Unlike usual likelihood-based methods for fitting mixtures, NPMLEs are based on convex optimization. We prove finite sample results on the Hellinger accuracy of every NPMLE. Our results imply, in particular, that every NPMLE achieves near parametric risk (up to logarithmic multiplicative factors) when the true density is a discrete Gaussian mixture without any prior information on the number of mixture components. NPMLEs can naturally be used to yield empirical Bayes estimates of the Oracle Bayes estimator in the Gaussian denoising problem. We prove bounds for the accuracy of the empirical Bayes estimate as an approximation to the Oracle Bayes estimator. Here our results imply that the empirical Bayes estimator performs at nearly the optimal level (up to logarithmic multiplicative factors) for denoising in clustering situations without any prior knowledge of the number of clusters.

Full work available at URL: https://arxiv.org/abs/1712.02009

Recommendations

• Adaptive density estimation for clustering with Gaussian mixtures
• A Smooth Nonparametric Estimate of a Mixing Distribution Using Mixtures of Gaussians
• Generalized maximum likelihood estimation of normal mixture densities
• Approximate nonparametric maximum likelihood for mixture models: a convex optimization approach to fitting arbitrary multivariate mixing distributions
• Maximum likelihood estimation of a class of non-Gaussian densities with application to I/sub p/ deconvolution

zbMATH Keywords

model selectiondensity estimationrate of convergenceconvex optimizationGaussian mixture modeladaptive estimationconvex clusteringHellinger distancemetric entropy

Mathematics Subject Classification ID

Density estimation (62G07) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Bayesian problems; characterization of Bayes procedures (62C10) Empirical decision procedures; empirical Bayes procedures (62C12)

Cites Work

• Estimating the number of clusters in a data set via the gap statistic
• Weak convergence and empirical processes. With applications to statistics
• Statistical analysis of finite mixture distributions
• Title not available (Why is that?)
• Finite mixture models
• Estimation of the mean of a multivariate normal distribution
• Mixture models: theory, geometry and applications
• The geometry of mixture likelihoods: A general theory
• Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities.
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• General maximum likelihood empirical Bayes estimation of normal means
• Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means
• Consistency of the Maximum Likelihood Estimator in the Presence of Infinitely Many Incidental Parameters
• Nonparametric Maximum Likelihood Estimation of a Mixing Distribution
• Title not available (Why is that?)
• Convex Optimization, Shape Constraints, Compound Decisions, and Empirical Bayes Rules
• Risk bounds for model selection via penalization
• Statistical guarantees for the EM algorithm: from population to sample-based analysis
• Title not available (Why is that?)
• Title not available (Why is that?)
• Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
• Medical applications of finite mixture models
• Tweedie’s Formula and Selection Bias
• Title not available (Why is that?)
• A non asymptotic penalized criterion for Gaussian mixture model selection
• A review of reliable maximum likelihood algorithms for semiparametric mixture models
• Data-driven penalty calibration: A case study for Gaussian mixture model selection
• Combinatorial methods in density estimation
• The geometry of mixture likelihoods, part II: The exponential family
• Multivariate empirical Bayes and estimation of covariance matrices
• Title not available (Why is that?)
• Probability inequalities for likelihood ratios and convergence rates of sieve MLEs
• SURE Estimates for a Heteroscedastic Hierarchical Model
• Empirical Bayes on vector observations: An extension of Stein's method
• The Empirical Bayes Approach to Statistical Decision Problems
• Title not available (Why is that?)
• Admissible Estimators, Recurrent Diffusions, and Insoluble Boundary Value Problems
• Adaptive density estimation for clustering with Gaussian mixtures
• The EM Algorithm and Related Statistical Models
• A review of semiparametric mixture models
• High-dimensional classification via nonparametric empirical Bayes and maximum likelihood inference
• Statistical properties of convex clustering
• Group-Linear Empirical Bayes Estimates for a Heteroscedastic Normal Mean
• Efficient density estimation via piecewise polynomial approximation
• Sample-Optimal Density Estimation in Nearly-Linear Time
• Sparse Convex Clustering
• Learning mixtures of structured distributions over discrete domains
• A new algorithm and theory for penalized regression-based clustering

Cited In (25)

• Maximum likelihood estimation of a class of non-Gaussian densities with application to I/sub p/ deconvolution
• Generalized maximum likelihood estimation of the mean of parameters of mixtures. With applications to sampling and to observational studies
• Set-convergence and its application: a tutorial
• A nonparametric empirical Bayes approach to large-scale multivariate regression
• Non-parametric likelihood based channel estimator for Gaussian mixture noise
• Minimax bounds for estimating multivariate Gaussian location mixtures
• Uniform consistency in nonparametric mixture models
• Fisher-Pitman Permutation Tests Based on Nonparametric Poisson Mixtures with Application to Single Cell Genomics
• Entropy regularization in probabilistic clustering
• Empirical Bayes Mean Estimation With Nonparametric Errors Via Order Statistic Regression on Replicated Data
• Nonparametric empirical Bayes biomarker imputation and estimation
• A Compound Decision Approach to Covariance Matrix Estimation
• High dimensional discriminant rules with shrinkage estimators of the covariance matrix and mean vector
• Simultaneous estimation of normal means with side information
• Learning Gaussian mixtures using the Wasserstein-Fisher-Rao gradient flow
• Poisson mean vector estimation with nonparametric maximum likelihood estimation and application to protein domain data
• No need for an oracle: the nonparametric maximum likelihood decision in the compound decision problem is minimax
• High-dimensional linear discriminant analysis using nonparametric methods
• Title not available (Why is that?)
• Likelihood Maximization and Moment Matching in Low <scp>SNR</scp> Gaussian Mixture Models
• Conditional Expected Likelihood Technique for Compound Gaussian and Gaussian Distributed Noise Mixtures
• Optimal estimation of high-dimensional Gaussian location mixtures
• Implicit models of Gaussian mixture densities and locally optimum detectors
• Invidious comparisons: ranking and selection as compound decisions
• Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in \(O(\sqrt{n})\) iterations

Uses Software

• Mosek
• EBayesThresh
• clusterpath

This page was built for publication: On the nonparametric maximum likelihood estimator for Gaussian location mixture densities with application to Gaussian denoising

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2196192)