A globally convergent algorithm for Lasso-penalized mixture of linear regression models
From MaRDI portal
Publication:1662084
DOI10.1016/J.CSDA.2017.09.003zbMATH Open1469.62109OpenAlexW2460444101MaRDI QIDQ1662084FDOQ1662084
Abstract: Variable selection is an old and pervasive problem in regression analysis. One solution is to impose a lasso penalty to shrink parameter estimates toward zero and perform continuous model selection. The lasso-penalized mixture of linear regressions model (L-MLR) is a class of regularization methods for the model selection problem in the fixed number of variables setting. In this article, we propose a new algorithm for the maximum penalized-likelihood estimation of the L-MLR model. This algorithm is constructed via the minorization--maximization algorithm paradigm. Such a construction allows for coordinate-wise updates of the parameter components, and produces globally convergent sequences of estimates that generate monotonic sequences of penalized log-likelihood values. These three features are missing in the previously presented approximate expectation-maximization algorithms. The previous difficulty in producing a globally convergent algorithm for the maximum penalized-likelihood estimation of the L-MLR model is due to the intractability of finding exact updates for the mixture model mixing proportions in the maximization-step. In our algorithm, we solve this issue by showing that it can be converted into a polynomial root finding problem. Our solution to this problem involves a polynomial basis conversion that is interesting in its own right. The method is tested in simulation and with an application to Major League Baseball salary data from the 1990s and the present day. We explore the concept of whether player salaries are associated with batting performance.
Recommendations
- On mixture regression shrinkage and selection via the MR-LASSO
- \(\ell_{1}\)-penalization for mixture regression models
- Variable selection using MM algorithms
- Coordinate descent algorithms for lasso penalized regression
- A penalized maximum likelihood approach for estimating mixture of regression models
Cites Work
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 1742537 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- A New Approach to Estimating Switching Regressions
- A Simplex Method for Function Minimization
- A constrained formulation of maximum-likelihood estimation for normal mixture distributions
- A maximum likelihood methodology for clusterwise linear regression
- A proof of convergence of the concave-convex procedure using Zangwill's theory
- A unified convergence analysis of block successive minimization methods for nonsmooth optimization
- An overview of the new feature selection methods in finite mixture of regression models
- Constrained monotone EM algorithms for finite mixture of multivariate Gaussians
- Coordinate descent algorithms for lasso penalized regression
- Degeneracy of the EM algorithm for the MLE of multivariate Gaussian mixtures and dynamic constraints
- Estimating the dimension of a model
- Finite mixture models
- Fitting finite mixtures of generalized linear regressions in \textsf{R}
- Identifiability of models for clusterwise linear regression
- Inference for normal mixtures in mean and variance
- Laplace mixture of linear experts
- Maximum Likelihood Estimation of Misspecified Models
- Maximum likelihood estimation of Gaussian mixture models without matrix operations
- Mixtures of linear regressions
- Model Selection and Estimation in Regression with Grouped Variables
- Modern Multivariate Statistical Techniques
- Multi-species distribution modeling using penalized mixture of regressions
- New estimation and feature selection methods in mixture-of-experts models
- Optimization
- Regularization in finite mixture of regression models with diverging number of parameters
- Sequential Minimax Search for a Maximum
- Set-valued optimization. An introduction with applications
- Sharp quadratic majorization in one dimension
- Statistical Analysis of Financial Data in S-Plus
- Statistical modeling: The two cultures. (With comments and a rejoinder).
- Statistics for high-dimensional data. Methods, theory and applications.
- The Variable Selection Problem
- Using mixtures in econometric models: a brief review and some new results
- Variable Selection in Finite Mixture of Regression Models
- Variable selection using MM algorithms
- \(\ell_{1}\)-penalization for mixture regression models
Cited In (9)
- An extended linearized alternating direction method of multipliers for fused-Lasso penalized linear regression
- Prediction of the Nash through penalized mixture of logistic regression models
- LASSO–penalized clusterwise linear regression modelling: a two–step approach
- Robust structured heterogeneity analysis approach for high-dimensional data
- Regression‐based heterogeneity analysis to identify overlapping subgroup structure in high‐dimensional data
- Local Linear Convergence of ISTA and FISTA on the LASSO Problem
- Variable Selection Using a Smooth Information Criterion for Distributional Regression Models
- Structured analysis of the high-dimensional FMR model
- Stochastic proximal-gradient algorithms for penalized mixed models
This page was built for publication: A globally convergent algorithm for Lasso-penalized mixture of linear regression models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1662084)
