Moment Estimation for Nonparametric Mixture Models through Implicit Tensor Decomposition

DOI10.1137/22M153879XzbMATH Open1530.62011arXiv2210.14386OpenAlexW4389451766MaRDI QIDQ6148355FDOQ6148355

Authors: Yifan Zhang, Joe Kileel 

Publication date: 11 January 2024

Published in: SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Abstract: We present an alternating least squares type numerical optimization scheme to estimate conditionally-independent mixture models in $mathbb{R}^n$, without parameterizing the distributions. Following the method of moments, we tackle an incomplete tensor decomposition problem to learn the mixing weights and componentwise means. Then we compute the cumulative distribution functions, higher moments and other statistics of the component distributions through linear solves. Crucially for computations in high dimensions, the steep costs associated with high-order tensors are evaded, via the development of efficient tensor-free operations. Numerical experiments demonstrate the competitive performance of the algorithm, and its applicability to many models and applications. Furthermore we provide theoretical analyses, establishing identifiability from low-order moments of the mixture and guaranteeing local linear convergence of the ALS algorithm.

Full work available at URL: https://arxiv.org/abs/2210.14386

Recommendations

• Tensor decomposition for learning Gaussian mixtures from moments
• Estimating Higher-Order Moments Using Symmetric Tensor Decomposition
• Tensor decompositions for learning latent variable models
• Overcomplete order-3 tensor decomposition, blind deconvolution, and Gaussian mixture models
• An operator theoretic approach to nonparametric mixture models

zbMATH Keywords

momentsnonparametric mixturenumerical optimizationtensor decomposition

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Nonparametric estimation (62G05) Estimation in multivariate analysis (62H12) Multilinear algebra, tensor calculus (15A69)

Cites Work

• Finite mixture models
• Title not available (Why is that?)
• Identifiability of parameters in latent structure models with many observed variables
• Nonparametric estimation of component distributions in a multivariate mixture
• Nonparametric mixture models with conditionally independent multivariate component densities
• Estimation of the number of components of nonparametric multivariate finite mixture models
• Semi-parametric estimation for conditional independence multivariate finite mixture models
• Maximum smoothed likelihood for multivariate mixtures
• Anderson acceleration for fixed-point iterations
• Nonparametric Estimation of Multivariate Mixtures
• Bandwidth selection in an EM-like algorithm for nonparametric multivariate mixtures
• Non-Parametric Identification and Estimation of the Number of Components in Multivariate Mixtures
• Statistical guarantees for the EM algorithm: from population to sample-based analysis
• Title not available (Why is that?)
• Editorial: Recent developments in mixture models (Hamburg, July 2001)
• On generic identifiability of symmetric tensors of subgeneric rank
• Nonparametric inference in multivariate mixtures
• Nonparametric Identification of Finite Mixture Models of Dynamic Discrete Choices
• The Numerical Solution of Systems of Polynomials Arising in Engineering and Science
• Identification of first-price auctions with non-separable unobserved heterogeneity
• Title not available (Why is that?)
• Identification and estimation of incomplete information games with multiple equilibria
• Identification of games of incomplete information with multiple equilibria and unobserved heterogeneity
• Nonparametric identification of dynamic models with unobserved state variables
• Learning mixtures of spherical Gaussians: moment methods and spectral decompositions (extended abstract)
• Non-Parametric Estimation of Finite Mixtures from Repeated Measurements
• Title not available (Why is that?)
• Learning diagonal Gaussian mixture models and incomplete tensor decompositions
• Tensor decomposition for learning Gaussian mixtures from moments
• Beyond the low-degree algorithm: mixtures of subcubes and their applications
• Estimating Higher-Order Moments Using Symmetric Tensor Decomposition
• Moment Varieties for Mixtures of Products

This page was built for publication: Moment Estimation for Nonparametric Mixture Models through Implicit Tensor Decomposition

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6148355)