Safe Feature Elimination in Sparse Supervised Learning
From MaRDI portal
Abstract: We investigate fast methods that allow to quickly eliminate variables (features) in supervised learning problems involving a convex loss function and a -norm penalty, leading to a potentially substantial reduction in the number of variables prior to running the supervised learning algorithm. The methods are not heuristic: they only eliminate features that are {em guaranteed} to be absent after solving the learning problem. Our framework applies to a large class of problems, including support vector machine classification, logistic regression and least-squares. The complexity of the feature elimination step is negligible compared to the typical computational effort involved in the sparse supervised learning problem: it grows linearly with the number of features times the number of examples, with much better count if data is sparse. We apply our method to data sets arising in text classification and observe a dramatic reduction of the dimensionality, hence in computational effort required to solve the learning problem, especially when very sparse classifiers are sought. Our method allows to immediately extend the scope of existing algorithms, allowing us to run them on data sets of sizes that were out of their reach before.
Recommendations
- Simultaneous Safe Feature and Sample Elimination for Sparse Support Vector Regression
- Safe feature elimination for non-negativity constrained convex optimization
- A safe reinforced feature screening strategy for Lasso based on feasible solutions
- Safe feature screening rules for the regularized Huber regression
- SAFE: An efficient feature extraction technique
- Sequential safe feature elimination rule for L₁-regularized regression with Kullback-Leibler divergence
- Scaling up sparse support vector machines by simultaneous feature and sample reduction
- Sparse supervised dimension reduction in high dimensional classification
Cited in
(53)- Natural coordinate descent algorithm for \(\ell_1\)-penalised regression in generalised linear models
- Double fused Lasso regularized regression with both matrix and vector valued predictors
- Regularization parameter selection for the low rank matrix recovery
- A hybrid acceleration strategy for nonparallel support vector machine
- Distance metric learning for graph structured data
- A decomposition method for Lasso problems with zero-sum constraint
- Screening for a reweighted penalized conditional gradient method
- On the distribution, model selection properties and uniqueness of the Lasso estimator in low and high dimensions
- Scaling up twin support vector regression with safe screening rule
- A safe reinforced feature screening strategy for Lasso based on feasible solutions
- An active-set proximal-Newton algorithm for \(\ell_1\) regularized optimization problems with box constraints
- Solving a class of feature selection problems via fractional 0--1 programming
- Safe feature elimination for non-negativity constrained convex optimization
- A safe screening rule for accelerating weighted twin support vector machine
- Concise comparative summaries (CCS) of large text corpora with a human experiment
- Safe feature screening rules for the regularized Huber regression
- Fast stepwise regression based on multidimensional indexes
- Optimization in high dimensions via accelerated, parallel, and proximal coordinate descent
- Thresholding least-squares inference in high-dimensional regression models
- Accelerated, parallel, and proximal coordinate descent
- Graphical Lasso and thresholding: equivalence and closed-form solutions
- Gap safe screening rules for sparsity enforcing penalties
- Understanding large text corpora via sparse machine learning
- Conducting sparse feature selection on arbitrarily long phrases in text corpora with a focus on interpretability
- Screening rules and its complexity for active set identification
- scientific article; zbMATH DE number 7626751 (Why is no real title available?)
- A Scalable Hierarchical Lasso for Gene–Environment Interactions
- A Pliable Lasso
- Adaptive hybrid screening for efficient lasso optimization
- Estimation of semiparametric regression model with right-censored high-dimensional data
- Dual extrapolation for sparse GLMs
- The sliding Frank-Wolfe algorithm and its application to super-resolution microscopy
- Scaling up sparse support vector machines by simultaneous feature and sample reduction
- Two-layer feature reduction for sparse-group Lasso via decomposition of convex sets
- Safe triplet screening for distance metric learning
- Safe Rules for the Identification of Zeros in the Solutions of the SLOPE Problem
- ``FISTA in Banach spaces with adaptive discretisations
- A novel ramp loss-based multi-task twin support vector machine with multi-parameter safe acceleration
- scientific article; zbMATH DE number 7750672 (Why is no real title available?)
- Smooth over-parameterized solvers for non-smooth structured optimization
- Proximal gradient/semismooth Newton methods for projection onto a polyhedron via the duality-gap-active-set strategy
- A safe double screening strategy for elastic net support vector machine
- Algorithms for Sparse Support Vector Machines
- Sequential safe feature elimination rule for L₁-regularized regression with Kullback-Leibler divergence
- Feature screening strategy for non-convex sparse logistic regression with log sum penalty
- Cardinality-constrained structured data-fitting problems
- Locally simultaneous inference
- Structure estimation of binary graphical models on stratified data: application to the description of injury tables for victims of road accidents
- Fast Lasso-type safe screening for Fine-Gray competing risks model with ultrahigh dimensional covariates
- Safe feature identification rule for fused Lasso by an extra dual variable
- Adaptive sieving: a dimension reduction technique for sparse optimization problems
- Nonsmoothness in machine learning: specific structure, proximal identification, and applications
- Sparse identification of posynomial models
This page was built for publication: Safe Feature Elimination in Sparse Supervised Learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4906144)
