Regularized Estimation and Feature Selection in Mixtures of Gaussian-Gated Experts Models

DOI10.1007/978-981-15-1960-4_3MaRDI QIDQ3305484zbMATH OpenOpenAlexFDO

Authors Faicel Chamroukhi, Florian Lecocq, Hien D. Nguyen

Publication date 7 August 2020

Published in Communications in Computer and Information Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1909.05494

zbMATH Keywords

clusteringfeature selectionLassoEM algorithmhigh-dimensional datamixtures-of-experts

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Statistical aspects of big data and data science (62R07) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: Mixtures-of-Experts models and their maximum likelihood estimation (MLE) via the EM algorithm have been thoroughly studied in the statistics and machine learning literature. They are subject of a growing investigation in the context of modeling with high-dimensional predictors with regularized MLE. We examine MoE with Gaussian gating network, for clustering and regression, and propose an $el{l}_1$-regularized MLE to encourage sparse models and deal with the high-dimensional setting. We develop an EM-Lasso algorithm to perform parameter estimation and utilize a BIC-like criterion to select the model parameters, including the sparsity tuning hyperparameters. Experiments conducted on simulated data show the good performance of the proposed regularized MLE compared to the standard MLE with the EM algorithm.

Recommendations

• Regularized estimation and feature selection in mixtures of experts
• New estimation and feature selection methods in mixture-of-experts models
• Regularization and selection in Gaussian mixture of autoregressive models
• RBF nets, mixture experts, and Bayesian Ying-Yang learning
• AI 2005: Advances in Artificial Intelligence

Cites work

• scientific article; zbMATH DE number 3567782 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 6438182 (Why is no real title available?)
• Coordinate descent algorithms for lasso penalized regression
• Finite mixture models
• Laplace mixture of linear experts
• New estimation and feature selection methods in mixture-of-experts models
• Pathwise coordinate optimization
• Penalized model-based clustering with application to variable selection
• Regularized estimation and feature selection in mixtures of experts
• Rejoinder to the comments on: \(\ell _{1}\)-penalization for mixture regression models
• Robust mixture of experts modeling using the \(t\) distribution
• The EM Algorithm and Extensions, 2E
• Time series modeling by a regression approach based on a latent process

Cited in

(4)

• Regularized parameter estimation in high-dimensional Gaussian mixture models
• New estimation and feature selection methods in mixture-of-experts models
• Regularized estimation and feature selection in mixtures of experts
• Functional mixtures-of-experts

This page was built for publication: Regularized Estimation and Feature Selection in Mixtures of Gaussian-Gated Experts Models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3305484)