Gap safe screening rules for sparsity enforcing penalties
From MaRDI portal
(Redirected from Gap safe screening rules for sparsity enforcing penalties)
Abstract: In high dimensional regression settings, sparsity enforcing penalties have proved useful to regularize the data-fitting term. A recently introduced technique called screening rules propose to ignore some variables in the optimization leveraging the expected sparsity of the solutions and consequently leading to faster solvers. When the procedure is guaranteed not to discard variables wrongly the rules are said to be safe. In this work, we propose a unifying framework for generalized linear models regularized with standard sparsity enforcing penalties such as or norms. Our technique allows to discard safely more variables than previously considered safe rules, particularly for low regularization parameters. Our proposed Gap Safe rules (so called because they rely on duality gap computation) can cope with any iterative solver but are particularly well suited to (block) coordinate descent methods. Applied to many standard learning tasks, Lasso, Sparse-Group Lasso, multi-task Lasso, binary and multinomial logistic regression, etc., we report significant speed-ups compared to previously proposed safe rules on all tested data sets.
Recommendations
- Lasso screening rules via dual polytope projection
- Strong Rules for Discarding Predictors in Lasso-Type Problems
- Hybrid safe-strong rules for efficient optimization in Lasso-type problems
- Screening rules and its complexity for active set identification
- Safe Feature Elimination in Sparse Supervised Learning
Cites work
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A first-order primal-dual algorithm for convex problems with applications to imaging
- An interior-point method for large-scale l₁-regularized logistic regression
- C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework
- Convex analysis and monotone operator theory in Hilbert spaces
- Convex multi-task feature learning
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Dynamic Screening: Accelerating First-Order Algorithms for the Lasso and Group-Lasso
- Gap safe screening rules for sparsity enforcing penalties
- scientific article; zbMATH DE number 439380 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Lasso screening rules via dual polytope projection
- Least angle regression. (With discussion)
- LIBLINEAR: a library for large linear classification
- Model Selection and Estimation in Regression with Grouped Variables
- Optimization with sparsity-inducing penalties
- Pathwise coordinate optimization
- Proximal methods for hierarchical sparse coding
- Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer
- Safe Feature Elimination in Sparse Supervised Learning
- Statistics for high-dimensional data. Methods, theory and applications.
- Strong Rules for Discarding Predictors in Lasso-Type Problems
- Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
- The Lasso problem and uniqueness
Cited in
(39)- Regularization parameter selection for the low rank matrix recovery
- A hybrid acceleration strategy for nonparallel support vector machine
- Safe sample screening rules for multicategory angle-based support vector machines
- Screening for a reweighted penalized conditional gradient method
- On the distribution, model selection properties and uniqueness of the Lasso estimator in low and high dimensions
- Communication-efficient distributed multi-task learning with matrix sparsity regularization
- A safe reinforced feature screening strategy for Lasso based on feasible solutions
- Perspective maximum likelihood-type estimation via proximal decomposition
- An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems
- Safe feature elimination for non-negativity constrained convex optimization
- Safe feature screening rules for the regularized Huber regression
- Graphical Lasso and thresholding: equivalence and closed-form solutions
- Gap safe screening rules for sparsity enforcing penalties
- Screening rules and its complexity for active set identification
- scientific article; zbMATH DE number 7626751 (Why is no real title available?)
- High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
- Dual extrapolation for sparse GLMs
- Solving the OSCAR and SLOPE models using a semismooth Newton-based augmented Lagrangian method
- Scaling up sparse support vector machines by simultaneous feature and sample reduction
- Safe triplet screening for distance metric learning
- Lasso screening rules via dual polytope projection
- Safe Rules for the Identification of Zeros in the Solutions of the SLOPE Problem
- ``FISTA in Banach spaces with adaptive discretisations
- scientific article; zbMATH DE number 7750672 (Why is no real title available?)
- High-Dimensional Portfolio Selection with Cardinality Constraints
- Smooth over-parameterized solvers for non-smooth structured optimization
- Proximal gradient/semismooth Newton methods for projection onto a polyhedron via the duality-gap-active-set strategy
- Sequential safe feature elimination rule for L₁-regularized regression with Kullback-Leibler divergence
- Feature screening strategy for non-convex sparse logistic regression with log sum penalty
- Variable screening for Lasso based on multidimensional indexing
- Cardinality-constrained structured data-fitting problems
- Techniques for accelerating branch-and-bound algorithms dedicated to sparse optimization
- Fast Lasso-type safe screening for Fine-Gray competing risks model with ultrahigh dimensional covariates
- Safe feature identification rule for fused Lasso by an extra dual variable
- Super-resolved Lasso
- Off-the-grid regularisation for Poisson inverse problems
- Self-concordant smoothing in proximal quasi-Newton algorithms for large-scale convex composite optimization
- Nonsmoothness in machine learning: specific structure, proximal identification, and applications
- Hybrid safe-strong rules for efficient optimization in Lasso-type problems
This page was built for publication: Gap safe screening rules for sparsity enforcing penalties
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4637055)
