Solving the OSCAR and SLOPE models using a semismooth Newton-based augmented Lagrangian method
From MaRDI portal
Abstract: The octagonal shrinkage and clustering algorithm for regression (OSCAR), equipped with the -norm and a pair-wise -norm regularizer, is a useful tool for feature selection and grouping in high-dimensional data analysis. The computational challenge posed by OSCAR, for high dimensional and/or large sample size data, has not yet been well resolved due to the non-smoothness and inseparability of the regularizer involved. In this paper, we successfully resolve this numerical challenge by proposing a sparse semismooth Newton-based augmented Lagrangian method to solve the more general SLOPE (the sorted L-one penalized estimation) model. By appropriately exploiting the inherent sparse and low-rank property of the generalized Jacobian of the semismooth Newton system in the augmented Lagrangian subproblem, we show how the computational complexity can be substantially reduced. Our algorithm presents a notable advantage in the high-dimensional statistical regression settings. Numerical experiments are conducted on real data sets, and the results demonstrate that our algorithm is far superior, in both speed and robustness, than the existing state-of-the-art algorithms based on first-order iterative schemes, including the widely used accelerated proximal gradient (APG) method and the alternating direction method of multipliers (ADMM).
Recommendations
- A highly efficient semismooth Newton augmented Lagrangian method for solving lasso problems
- An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems
- Efficient sparse semismooth Newton methods for the clustered Lasso problem
- A dual semismooth Newton based augmented Lagrangian method for large-scale linearly constrained sparse group square-root Lasso problems
- A dual based semismooth Newton-type algorithm for solving large-scale sparse Tikhonov regularization problems
Cites work
- A Biometrics Invited Paper. The Analysis and Selection of Variables in Linear Regression
- A dual algorithm for the solution of nonlinear variational problems via finite element approximation
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A highly efficient semismooth Newton augmented Lagrangian method for solving lasso problems
- A nonsmooth version of Newton's method
- Adaptive Lasso for sparse high-dimensional regression models
- An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems
- An efficient inexact symmetric Gauss-Seidel based majorized ADMM for high-dimensional convex composite conic programming
- Atomic Decomposition by Basis Pursuit
- Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming
- Constrained Statistical Inference
- Convex Analysis
- Gap safe screening rules for sparsity enforcing penalties
- Hankel matrix rank minimization with applications to system identification and realization
- scientific article; zbMATH DE number 446509 (Why is no real title available?)
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 4082855 (Why is no real title available?)
- scientific article; zbMATH DE number 193111 (Why is no real title available?)
- scientific article; zbMATH DE number 3574917 (Why is no real title available?)
- scientific article; zbMATH DE number 1906319 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Just relax: convex programming methods for identifying sparse signals in noise
- Monotone Operators and the Proximal Point Algorithm
- Newton and quasi-Newton methods for normal maps with polyhedral sets
- On efficiently solving the subproblems of a level-set method for fused lasso problems
- On the Moreau-Yosida regularization of the vector k-norm related functions
- On the R-superlinear convergence of the KKT residuals generated by the augmented Lagrangian method for convex composite conic programming
- Pathwise coordinate optimization
- Proximité et dualité dans un espace hilbertien
- Semismooth and Semiconvex Functions in Constrained Optimization
- Semismooth Matrix-Valued Functions
- Set-valued analysis
- Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR
- SLOPE-adaptive variable selection via convex optimization
- Some continuity properties of polyhedral multifunctions
- Sparse modeling for image and vision processing
- The Isotonic Regression Problem and Its Dual
- Variational Analysis
Cited in
(16)- An efficient Hessian based algorithm for singly linearly and box constrained least squares regression
- A Lagrange-Newton algorithm for sparse nonlinear programming
- A semismooth Newton method for support vector classification and regression
- A highly efficient semismooth Newton augmented Lagrangian method for solving lasso problems
- Efficient sparse Hessian-based semismooth Newton algorithms for Dantzig selector
- An efficient augmented Lagrangian method for support vector machine
- Efficient sparse semismooth Newton methods for the clustered Lasso problem
- B-subdifferentials of the projection onto the generalized simplex
- Safe Rules for the Identification of Zeros in the Solutions of the SLOPE Problem
- A dual-based stochastic inexact algorithm for a class of stochastic nonsmooth convex composite problems
- An efficient sieving-based secant method for sparse optimization problems with least-squares constraints
- Efficient path algorithms for clustered Lasso and OSCAR
- Wasserstein distributionally robust optimization and its tractable regularization formulation
- Adaptive sieving: a dimension reduction technique for sparse optimization problems
- Multiple regression for matrix and vector predictors: models, theory, algorithms, and beyond
- The augmented Lagrangian methods: overview and recent advances
This page was built for publication: Solving the OSCAR and SLOPE models using a semismooth Newton-based augmented Lagrangian method
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5214191)
