_1-penalization for mixture regression models
From MaRDI portal
Publication:619141
Abstract: We consider a finite mixture of regressions (FMR) model for high-dimensional inhomogeneous data where the number of covariates may be much larger than sample size. We propose an l1-penalized maximum likelihood estimator in an appropriate parameterization. This kind of estimation belongs to a class of problems where optimization and theory for non-convex functions is needed. This distinguishes itself very clearly from high-dimensional estimation with convex loss- or objective functions, as for example with the Lasso in linear or generalized linear models. Mixture models represent a prime and important example where non-convexity arises. For FMR models, we develop an efficient EM algorithm for numerical optimization with provable convergence properties. Our penalized estimator is numerically better posed (e.g., boundedness of the criterion function) than unpenalized maximum likelihood estimation, and it allows for effective statistical regularization including variable selection. We also present some asymptotic theory and oracle inequalities: due to non-convexity of the negative log-likelihood function, different mathematical arguments are needed than for problems with convex losses. Finally, we apply the new method to both simulated and real data.
Recommendations
- Finite mixture regression: a sparse variable selection by model selection for clustering
- An \(\ell_{1}\)-oracle inequality for the Lasso in finite mixture Gaussian regression models
- A penalized maximum likelihood approach for estimating mixture of regression models
- Variable Selection in Finite Mixture of Regression Models
- On mixture regression shrinkage and selection via the MR-LASSO
Cites work
- A coordinate gradient descent method for nonsmooth separable minimization
- Adaptive Lasso for sparse high-dimensional regression models
- Asymptotic Statistics
- Convergence of a block coordinate descent method for nondifferentiable minimization
- Decoding by Linear Programming
- Finite mixture models
- Fitting finite mixtures of generalized linear regressions in \textsf{R}
- High-dimensional generalized linear models and the lasso
- High-dimensional graphs and variable selection with the Lasso
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 3826980 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 1206370 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 1420699 (Why is no real title available?)
- Lasso-type recovery of sparse representations for high-dimensional data
- Near-ideal model selection by \(\ell _{1}\) minimization
- Nearly unbiased variable selection under minimax concave penalty
- On Recovery of Sparse Signals Via $\ell _{1}$ Minimization
- On the conditions used to prove oracle results for the Lasso
- On the convergence properties of the EM algorithm
- Optimal aggregation of classifiers in statistical learning.
- Pathwise coordinate optimization
- Penalized model-based clustering with application to variable selection
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
- Simultaneous analysis of Lasso and Dantzig selector
- Some sharp performance bounds for least squares regression with L₁ regularization
- Sparsity oracle inequalities for the Lasso
- Stable Recovery of Sparse Signals and an Oracle Inequality
- The Adaptive Lasso and Its Oracle Properties
- The Bayesian Lasso
- The Dantzig selector and sparsity oracle inequalities
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The Group Lasso for Logistic Regression
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Variable Selection in Finite Mixture of Regression Models
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Weak convergence and empirical processes. With applications to statistics
Cited in
(only showing first 100 items - show all)- Pursuing Sources of Heterogeneity in Modeling Clustered Population
- Tree-Structured Clustering in Fixed Effects Models
- Asymptotic normality and optimalities in estimation of large Gaussian graphical models
- Robust Bayesian regularized estimation based on \(t\) regression model
- Bayesian variable selection for finite mixture model of linear regressions
- Wavelet-based scalar-on-function finite mixture regression models
- A globally convergent algorithm for Lasso-penalized mixture of linear regression models
- Inverse regression approach to robust nonlinear high-to-low dimensional mapping
- Ruin probabilities in the mixed claim frequency risk models
- Prediction with a flexible finite mixture-of-regressions
- Robust variable selection for finite mixture regression models
- Debiasing the Lasso: optimal sample size for Gaussian designs
- High-dimensional integrative analysis with homogeneity and sparsity recovery
- Compound Poisson point processes, concentration and oracle inequalities
- A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models
- On an extension of the promotion time cure model
- Estimating finite mixtures of ordinal graphical models
- Robust error density estimation in ultrahigh dimensional sparse linear model
- A convex optimization framework for the identification of homogeneous reaction systems
- Adaptive estimation in the supremum norm for semiparametric mixtures of regressions
- In the pursuit of sparseness: a new rank-preserving penalty for a finite mixture of factor analyzers
- Prediction of the Nash through penalized mixture of logistic regression models
- Robust subspace clustering
- Pivotal estimation via square-root lasso in nonparametric regression
- Perspective maximum likelihood-type estimation via proximal decomposition
- Robust finite mixture regression for heterogeneous targets
- Variance prior forms for high-dimensional Bayesian variable selection
- A latent discrete Markov random field approach to identifying and classifying historical forest communities based on spatial multivariate tree species counts
- Non-concave penalization in linear mixed-effect models and regularized selection of fixed effects
- Bayesian variable selection in linear regression models with non-normal errors
- Stochastic proximal-gradient algorithms for penalized mixed models
- Structured regularization for conditional Gaussian graphical models
- Joint rank and variable selection for parsimonious estimation in a high-dimensional finite mixture regression model
- Model-based regression clustering for high-dimensional data: application to functional data
- Penalized estimation in high-dimensional hidden Markov models with state-specific graphical models
- Endogeneity in high dimensions
- Maximin effects in inhomogeneous large-scale data
- Penalised robust estimators for sparse and high-dimensional linear models
- Covariance matrix estimation of the maximum likelihood estimator in multivariate clusterwise linear regression
- Modeling cell populations measured by flow cytometry with covariates using sparse mixture of regressions
- An _1-oracle inequality for the Lasso in multivariate finite mixture of multivariate Gaussian regression models
- Regularization in finite mixture of regression models with diverging number of parameters
- Censored linear model in high dimensions. Penalised linear regression on high-dimensional data with left-censored response variable
- Estimation for high-dimensional linear mixed-effects models using _1-penalization
- Regularized parameter estimation in high-dimensional Gaussian mixture models
- Joint estimation of precision matrices in heterogeneous populations
- On estimation of the diagonal elements of a sparse precision matrix
- The benefit of group sparsity in group inference with de-biased scaled group Lasso
- A partially linear framework for massive heterogeneous data
- Semiparametric estimation of a two-component mixture of linear regressions in which one component is known
- Minimum distance estimation in a finite mixture regression model
- A hierarchical Bayesian perspective on majorization-minimization for non-convex sparse regression: application to M/EEG source imaging
- Mixture of Gaussian regressions model with logistic weights, a penalized maximum likelihood approach
- Adapting to unknown noise level in sparse deconvolution
- A Sparse Learning Approach to Relative-Volatility-Managed Portfolio Selection
- Outlier detection and robust mixture modeling using nonconvex penalized likelihood
- A penalized maximum likelihood approach for estimating mixture of regression models
- Penalized proportion estimation for non parametric mixture of regressions
- A new model selection procedure for finite mixture regression models
- A robust high dimensional estimation of a finite mixture of the generalized linear model
- Penalized estimation in finite mixture of ultra-high dimensional regression models
- Bayesian variable selection in a finite mixture of linear mixed-effects models
- A New Semiparametric Approach to Finite Mixture of Regressions using Penalized Regression via Fusion
- Variable selection approach for zero-inflated count data via adaptive Lasso
- Model selection in finite mixture of regression models: a Bayesian approach with innovative weightedgpriors and reversible jump Markov chain Monte Carlo implementation
- A convex program for mixed linear regression with a recovery guarantee for well-separated data
- Screening and clustering of sparse regressions with finite non-Gaussian mixtures
- Mixtures, envelopes and hierarchical duality
- An \(\ell_{1}\)-oracle inequality for the Lasso in finite mixture Gaussian regression models
- Identification of sparse FIR systems using a general quantisation scheme
- Quasi-likelihood and/or robust estimation in high dimensions
- High-dimensional regression with unknown variance
- A general theory of concave regularization for high-dimensional sparse estimation problems
- Comments on: ``A random forest guided tour
- Histopathological imaging‐based cancer heterogeneity analysis via penalized fusion with model averaging
- Hierarchical cancer heterogeneity analysis based on histopathological imaging features
- Variance estimation in high-dimensional linear regression via adaptive elastic-net
- A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates
- Regression‐based heterogeneity analysis to identify overlapping subgroup structure in high‐dimensional data
- Regularization in dynamic random‐intercepts models for analysis of longitudinal data
- Heterogeneity Analysis via Integrating Multi-Sources High-Dimensional Data With Applications to Cancer Studies
- Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures
- Mixture of inhomogeneous matrix models for species‐rich ecosystems
- LASSO–penalized clusterwise linear regression modelling: a two–step approach
- Comments on: \(\ell_{1}\)-penalization for mixture regression models
- Estimation of multiple networks with common structures in heterogeneous subgroups
- Functional mixtures-of-experts
- Asymptotic bias of the _2-regularized error variance estimator
- Integrated subgroup identification from multi-source data
- Challenges in model-based clustering
- Clustering electricity consumers using high-dimensional regression mixture models
- Sparse estimation in semiparametric finite mixture of varying coefficient regression models
- Aggregated inference
- Information-incorporated sparse hierarchical cancer heterogeneity analysis
- Image-Based Prognostics Using Penalized Tensor Regression
- Mixture of Regression Models for Large Spatial Datasets
- Nonparametric Finite Mixture of Gaussian Graphical Models
- Bayesian hierarchical finite mixture of regression for histopathological imaging-based cancer data analysis
- Two-part quantile regression models for semi-continuous longitudinal data: a finite mixture approach
- Environmental risk assessment via nonhomogeneous hidden semi-Markov models with penalized vector autoregression
This page was built for publication: \(\ell_{1}\)-penalization for mixture regression models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q619141)
