A globally convergent algorithm for Lasso-penalized mixture of linear regression models
From MaRDI portal
Publication:1662084
DOI10.1016/J.CSDA.2017.09.003zbMATH Open1469.62109arXiv1603.08326OpenAlexW2460444101MaRDI QIDQ1662084FDOQ1662084
Authors: Luke R. Lloyd-Jones, Geoffrey J. McLachlan, Hien D. Nguyen 
Publication date: 17 August 2018
Published in: Computational Statistics and Data Analysis (Search for Journal in Brave)
Abstract: Variable selection is an old and pervasive problem in regression analysis. One solution is to impose a lasso penalty to shrink parameter estimates toward zero and perform continuous model selection. The lasso-penalized mixture of linear regressions model (L-MLR) is a class of regularization methods for the model selection problem in the fixed number of variables setting. In this article, we propose a new algorithm for the maximum penalized-likelihood estimation of the L-MLR model. This algorithm is constructed via the minorization--maximization algorithm paradigm. Such a construction allows for coordinate-wise updates of the parameter components, and produces globally convergent sequences of estimates that generate monotonic sequences of penalized log-likelihood values. These three features are missing in the previously presented approximate expectation-maximization algorithms. The previous difficulty in producing a globally convergent algorithm for the maximum penalized-likelihood estimation of the L-MLR model is due to the intractability of finding exact updates for the mixture model mixing proportions in the maximization-step. In our algorithm, we solve this issue by showing that it can be converted into a polynomial root finding problem. Our solution to this problem involves a polynomial basis conversion that is interesting in its own right. The method is tested in simulation and with an application to Major League Baseball salary data from the 1990s and the present day. We explore the concept of whether player salaries are associated with batting performance.
Full work available at URL: https://arxiv.org/abs/1603.08326
Recommendations
- On mixture regression shrinkage and selection via the MR-LASSO
- \(\ell_{1}\)-penalization for mixture regression models
- Variable selection using MM algorithms
- Coordinate descent algorithms for lasso penalized regression
- A penalized maximum likelihood approach for estimating mixture of regression models
Computational methods for problems pertaining to statistics (62-08) Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07) Applications of statistics (62P99)
Cites Work
- Fitting finite mixtures of generalized linear regressions in \textsf{R}
- Statistical Analysis of Financial Data in S-Plus
- Title not available (Why is that?)
- Estimating the dimension of a model
- Coordinate descent algorithms for lasso penalized regression
- Title not available (Why is that?)
- Finite mixture models
- Title not available (Why is that?)
- Statistics for high-dimensional data. Methods, theory and applications.
- Maximum Likelihood Estimation of Misspecified Models
- A constrained formulation of maximum-likelihood estimation for normal mixture distributions
- Model Selection and Estimation in Regression with Grouped Variables
- A Simplex Method for Function Minimization
- A New Approach to Estimating Switching Regressions
- Statistical modeling: The two cultures. (With comments and a rejoinder).
- Identifiability of models for clusterwise linear regression
- Variable selection using MM algorithms
- A unified convergence analysis of block successive minimization methods for nonsmooth optimization
- Modern Multivariate Statistical Techniques
- Optimization
- \(\ell_{1}\)-penalization for mixture regression models
- Sharp quadratic majorization in one dimension
- Constrained monotone EM algorithms for finite mixture of multivariate Gaussians
- Regularization in finite mixture of regression models with diverging number of parameters
- An overview of the new feature selection methods in finite mixture of regression models
- Inference for normal mixtures in mean and variance
- Variable Selection in Finite Mixture of Regression Models
- Multi-species distribution modeling using penalized mixture of regressions
- Mixtures of linear regressions
- Set-valued Optimization
- A maximum likelihood methodology for clusterwise linear regression
- Laplace mixture of linear experts
- New estimation and feature selection methods in mixture-of-experts models
- The Variable Selection Problem
- Sequential Minimax Search for a Maximum
- Degeneracy of the EM algorithm for the MLE of multivariate Gaussian mixtures and dynamic constraints
- Maximum likelihood estimation of Gaussian mixture models without matrix operations
- A proof of convergence of the concave-convex procedure using Zangwill's theory
- Using Mixtures in Econometric Models: A Brief Review and Some New Results
Cited In (9)
- An extended linearized alternating direction method of multipliers for fused-Lasso penalized linear regression
- Prediction of the Nash through penalized mixture of logistic regression models
- LASSO–penalized clusterwise linear regression modelling: a two–step approach
- Robust structured heterogeneity analysis approach for high-dimensional data
- Regression‐based heterogeneity analysis to identify overlapping subgroup structure in high‐dimensional data
- Local Linear Convergence of ISTA and FISTA on the LASSO Problem
- Variable Selection Using a Smooth Information Criterion for Distributional Regression Models
- Structured analysis of the high-dimensional FMR model
- Stochastic proximal-gradient algorithms for penalized mixed models
Uses Software
This page was built for publication: A globally convergent algorithm for Lasso-penalized mixture of linear regression models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1662084)
