A generic path algorithm for regularized statistical estimation
From MaRDI portal
Publication:4975409
Abstract: Regularization is widely used in statistics and machine learning to prevent overfitting and gear solution towards prior information. In general, a regularized estimation problem minimizes the sum of a loss function and a penalty term. The penalty term is usually weighted by a tuning parameter and encourages certain constraints on the parameters to be estimated. Particular choices of constraints lead to the popular lasso, fused-lasso, and other generalized penalized regression methods. Although there has been a lot of research in this area, developing efficient optimization methods for many nonseparable penalties remains a challenge. In this article we propose an exact path solver based on ordinary differential equations (EPSODE) that works for any convex loss function and can deal with generalized penalties as well as more complicated regularization such as inequality constraints encountered in shape-restricted regressions and nonparametric density estimation. In the path following process, the solution path hits, exits, and slides along the various constraints and vividly illustrates the tradeoffs between goodness of fit and model parsimony. In practice, the EPSODE can be coupled with AIC, BIC, or cross-validation to select an optimal tuning parameter. Our applications to generalized regularized generalized linear models, shape-restricted regressions, Gaussian graphical models, and nonparametric density estimation showcase the potential of the EPSODE algorithm.
Recommendations
- The solution path of the generalized lasso
- Piecewise linear regularized solution paths
- An ordinary differential equation-based solution path algorithm
- A New Algorithm in Maximum Likelihood Estimation for Generalized Linear Models
- Efficient global approximation of generalized nonlinear \(\ell _{1}\)-regularized solution paths and its applications
Cites work
- scientific article; zbMATH DE number 193111 (Why is no real title available?)
- scientific article; zbMATH DE number 1261669 (Why is no real title available?)
- scientific article; zbMATH DE number 1834445 (Why is no real title available?)
- scientific article; zbMATH DE number 3314854 (Why is no real title available?)
- $\ell_1$ Trend Filtering
- A new approach to variable selection in least squares problems
- A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
- Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
- An active set algorithm to estimate parameters in generalized linear models with ordered predictors
- An improved model averaging scheme for logistic regression
- An ordinary differential equation-based solution path algorithm
- Detecting the Presence of Mixing with Multiscale Maximum Likelihood
- Ideal spatial adaptation by wavelet shrinkage
- Inference and modeling with log-concave distributions
- Least angle regression. (With discussion)
- Limit distribution theory for maximum likelihood estimation of a log-concave density
- Numerical analysis for statisticians
- On the ``degrees of freedom of the lasso
- Parametrically guided generalised additive models with application to mergers and acquisitions data
- Piecewise linear regularized solution paths
- Quantile Regression in Reproducing Kernel Hilbert Spaces
- Sparse inverse covariance estimation with the graphical lasso
- Sparsity and Smoothness Via the Fused Lasso
- The solution path of the generalized lasso
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(19)- scientific article; zbMATH DE number 7415081 (Why is no real title available?)
- Efficient global approximation of generalized nonlinear \(\ell _{1}\)-regularized solution paths and its applications
- Path following in the exact penalty method of convex programming
- Solution path for quantile regression with epsilon-insensitive loss in a reproducing kernel Hilbert space
- An ordinary differential equation-based solution path algorithm
- Dynamic Visualization and Fast Computation for Convex Clustering via Algorithmic Regularization
- Model selection for high-dimensional quadratic regression via regularization
- Path-following methods for maximum a posteriori estimators in Bayesian hierarchical models: how estimates depend on hyperparameters
- Primal path algorithm for compositional data analysis
- Efficient computation for differential network analysis with applications to quadratic discriminant analysis
- scientific article; zbMATH DE number 7491400 (Why is no real title available?)
- L 1-Regularization Path Algorithm for Generalized Linear Models
- Global least squares path modeling: a full-information alternative to partial least squares path modeling
- Weak convergence of the regularization path in penalized M-estimation
- On the structure of regularization paths for piecewise differentiable regularization terms
- Solution paths for the generalized Lasso with applications to spatially varying coefficients regression
- LARS-type algorithm for group Lasso
- Algorithms for Fitting the Constrained Lasso
- Efficient regularized regression with \(L_0\) penalty for variable selection and network construction
This page was built for publication: A generic path algorithm for regularized statistical estimation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4975409)
