Moment Estimation for Nonparametric Mixture Models through Implicit Tensor Decomposition

DOI10.1137/22M153879XMaRDI QIDQ6148355zbMATH OpenOpenAlexFDO

Authors Yifan Zhang, Joe Kileel

Publication date 11 January 2024

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2210.14386

zbMATH Keywords

momentsnonparametric mixturenumerical optimizationtensor decomposition

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Nonparametric estimation (62G05) Estimation in multivariate analysis (62H12) Multilinear algebra, tensor calculus (15A69)

Abstract: We present an alternating least squares type numerical optimization scheme to estimate conditionally-independent mixture models in $mathbb{R}^n$, without parameterizing the distributions. Following the method of moments, we tackle an incomplete tensor decomposition problem to learn the mixing weights and componentwise means. Then we compute the cumulative distribution functions, higher moments and other statistics of the component distributions through linear solves. Crucially for computations in high dimensions, the steep costs associated with high-order tensors are evaded, via the development of efficient tensor-free operations. Numerical experiments demonstrate the competitive performance of the algorithm, and its applicability to many models and applications. Furthermore we provide theoretical analyses, establishing identifiability from low-order moments of the mixture and guaranteeing local linear convergence of the ALS algorithm.

Recommendations

• Tensor decomposition for learning Gaussian mixtures from moments
• Estimating Higher-Order Moments Using Symmetric Tensor Decomposition
• Tensor decompositions for learning latent variable models
• Overcomplete order-3 tensor decomposition, blind deconvolution, and Gaussian mixture models
• An operator theoretic approach to nonparametric mixture models

Cites work

• scientific article; zbMATH DE number 52497 (Why is no real title available?)
• scientific article; zbMATH DE number 1181673 (Why is no real title available?)
• scientific article; zbMATH DE number 2090249 (Why is no real title available?)
• scientific article; zbMATH DE number 2110454 (Why is no real title available?)
• Anderson acceleration for fixed-point iterations
• Bandwidth selection in an EM-like algorithm for nonparametric multivariate mixtures
• Beyond the low-degree algorithm: mixtures of subcubes and their applications
• Editorial: Recent developments in mixture models (Hamburg, July 2001)
• Estimating Higher-Order Moments Using Symmetric Tensor Decomposition
• Estimation of the number of components of nonparametric multivariate finite mixture models
• Finite mixture models
• Identifiability of parameters in latent structure models with many observed variables
• Identification and estimation of incomplete information games with multiple equilibria
• Identification of first-price auctions with non-separable unobserved heterogeneity
• Identification of games of incomplete information with multiple equilibria and unobserved heterogeneity
• Learning diagonal Gaussian mixture models and incomplete tensor decompositions
• Learning mixtures of spherical Gaussians: moment methods and spectral decompositions (extended abstract)
• Maximum smoothed likelihood for multivariate mixtures
• Moment Varieties for Mixtures of Products
• Non-Parametric Estimation of Finite Mixtures from Repeated Measurements
• Non-Parametric Identification and Estimation of the Number of Components in Multivariate Mixtures
• Nonparametric Estimation of Multivariate Mixtures
• Nonparametric Identification of Finite Mixture Models of Dynamic Discrete Choices
• Nonparametric estimation of component distributions in a multivariate mixture
• Nonparametric identification of dynamic models with unobserved state variables
• Nonparametric inference in multivariate mixtures
• Nonparametric mixture models with conditionally independent multivariate component densities
• On generic identifiability of symmetric tensors of subgeneric rank
• Semi-parametric estimation for conditional independence multivariate finite mixture models
• Statistical guarantees for the EM algorithm: from population to sample-based analysis
• Tensor decomposition for learning Gaussian mixtures from moments
• The Numerical Solution of Systems of Polynomials Arising in Engineering and Science

This page was built for publication: Moment Estimation for Nonparametric Mixture Models through Implicit Tensor Decomposition

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6148355)