Approximations of conditional probability density functions in Lebesgue spaces via mixture of experts models

DOI10.1186/S40488-021-00125-0zbMATH Open1490.62162arXiv2012.02385OpenAlexW3188130019WikidataQ114061064 ScholiaQ114061064MaRDI QIDQ2129248FDOQ2129248

Geoffrey J. McLachlan, Faicel Chamroukhi, TrungTin Nguyen, Hien D. Nguyen

Publication date: 22 April 2022

Published in: Journal of Statistical Distributions and Applications (Search for Journal in Brave)

Abstract: Mixture of experts (MoE) models are widely applied for conditional probability density estimation problems. We demonstrate the richness of the class of MoE models by proving denseness results in Lebesgue spaces, when inputs and outputs variables are both compactly supported. We further prove an almost uniform convergence result when the input is univariate. Auxiliary lemmas are proved regarding the richness of the soft-max gating function class, and their relationships to the class of Gaussian gating functions.

Full work available at URL: https://arxiv.org/abs/2012.02385

Recommendations

• Mixture of experts architectures for neural networks as a special case of conditional expectation formula.
• A universal approximation theorem for mixture-of-experts models
• Approximation of conditional densities by smooth mixtures of regressions
• Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation
• Error bounds for functional approximation and estimation using mixtures of experts

zbMATH Keywords

mixture modelsmixture of expertsapproximation theoryLebesgue spacesconditional probability density functions

Mathematics Subject Classification ID

Density estimation (62G07) Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Classification and discrimination; cluster analysis (statistical aspects) (62H30) General nonlinear regression (62J02)

Cites Work

• Inverse regression approach to robust nonlinear high-to-low dimensional mapping
• Gaussian mixture vector autoregression
• Local statistical modeling via a cluster-weighted approach with elliptical distributions
• Learning Theory
• Density estimation through convex combinations of densities: Approximation and estimation bounds
• Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation
• A distribution-free theory of nonparametric regression
• Title not available (Why is that?)
• High-dimensional regression with Gaussian mixtures and partially-latent response variables
• Risk bounds for mixture density estimation
• Variable Lebesgue spaces. Foundations and harmonic analysis
• Model-based clustering via linear cluster-weighted models
• Laplace mixture of linear experts
• An introductory course in Lebesgue spaces
• Title not available (Why is that?)
• Approximation of conditional densities by smooth mixtures of regressions
• Title not available (Why is that?)
• Finite mixture regression: a sparse variable selection by model selection for clustering
• Title not available (Why is that?)
• POSTERIOR CONSISTENCY IN CONDITIONAL DENSITY ESTIMATION BY COVARIATE DEPENDENT MIXTURES
• Mixture of Gaussian regressions model with logistic weights, a penalized maximum likelihood approach
• Robust mixture of experts modeling using the \(t\) distribution
• Nonparametric Regression Estimation by Normalized Radial Basis Function Networks
• Error bounds for functional approximation and estimation using mixtures of experts
• A Universal Approximation Theorem for Mixture-of-Experts Models
• Bayesian regression with heteroscedastic error density and parametric mean function
• Approximation by finite mixtures of continuous density functions that vanish at infinity
• An \(\ell_1\)-oracle inequality for the Lasso in multivariate finite mixture of multivariate Gaussian regression models
• On convergence rates of mixtures of polynomial experts

Cited In (7)

• Dealing with overdispersion in multivariate count data
• Regression‐based heterogeneity analysis to identify overlapping subgroup structure in high‐dimensional data
• Summary statistics and discrepancy measures for approximate Bayesian computation via surrogate posteriors
• A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models
• Error bounds for functional approximation and estimation using mixtures of experts
• Approximation of probability density functions via location-scale finite mixtures in Lebesgue spaces
• Functional mixtures-of-experts

This page was built for publication: Approximations of conditional probability density functions in Lebesgue spaces via mixture of experts models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2129248)