Safe Feature Elimination in Sparse Supervised Learning
From MaRDI portal
Abstract: We investigate fast methods that allow to quickly eliminate variables (features) in supervised learning problems involving a convex loss function and a -norm penalty, leading to a potentially substantial reduction in the number of variables prior to running the supervised learning algorithm. The methods are not heuristic: they only eliminate features that are {em guaranteed} to be absent after solving the learning problem. Our framework applies to a large class of problems, including support vector machine classification, logistic regression and least-squares. The complexity of the feature elimination step is negligible compared to the typical computational effort involved in the sparse supervised learning problem: it grows linearly with the number of features times the number of examples, with much better count if data is sparse. We apply our method to data sets arising in text classification and observe a dramatic reduction of the dimensionality, hence in computational effort required to solve the learning problem, especially when very sparse classifiers are sought. Our method allows to immediately extend the scope of existing algorithms, allowing us to run them on data sets of sizes that were out of their reach before.
Recommendations
- Simultaneous Safe Feature and Sample Elimination for Sparse Support Vector Regression
- Safe feature elimination for non-negativity constrained convex optimization
- A safe reinforced feature screening strategy for Lasso based on feasible solutions
- Safe feature screening rules for the regularized Huber regression
- SAFE: An efficient feature extraction technique
- Sequential safe feature elimination rule for L₁-regularized regression with Kullback-Leibler divergence
- Scaling up sparse support vector machines by simultaneous feature and sample reduction
- Sparse supervised dimension reduction in high dimensional classification
Cited in
(53)- Scaling up sparse support vector machines by simultaneous feature and sample reduction
- Sparse identification of posynomial models
- Sequential safe feature elimination rule for L₁-regularized regression with Kullback-Leibler divergence
- Nonsmoothness in machine learning: specific structure, proximal identification, and applications
- A Pliable Lasso
- Understanding large text corpora via sparse machine learning
- Two-layer feature reduction for sparse-group Lasso via decomposition of convex sets
- Estimation of semiparametric regression model with right-censored high-dimensional data
- Cardinality-constrained structured data-fitting problems
- Smooth over-parameterized solvers for non-smooth structured optimization
- Adaptive hybrid screening for efficient lasso optimization
- Concise comparative summaries (CCS) of large text corpora with a human experiment
- Double fused Lasso regularized regression with both matrix and vector valued predictors
- On the distribution, model selection properties and uniqueness of the Lasso estimator in low and high dimensions
- The sliding Frank-Wolfe algorithm and its application to super-resolution microscopy
- Solving a class of feature selection problems via fractional 0--1 programming
- ``FISTA in Banach spaces with adaptive discretisations
- Gap safe screening rules for sparsity enforcing penalties
- Regularization parameter selection for the low rank matrix recovery
- Thresholding least-squares inference in high-dimensional regression models
- scientific article; zbMATH DE number 7750672 (Why is no real title available?)
- Safe triplet screening for distance metric learning
- Locally simultaneous inference
- Accelerated, parallel, and proximal coordinate descent
- Proximal gradient/semismooth Newton methods for projection onto a polyhedron via the duality-gap-active-set strategy
- A safe double screening strategy for elastic net support vector machine
- Safe Rules for the Identification of Zeros in the Solutions of the SLOPE Problem
- Algorithms for Sparse Support Vector Machines
- A Scalable Hierarchical Lasso for Gene–Environment Interactions
- Safe feature screening rules for the regularized Huber regression
- Natural coordinate descent algorithm for \(\ell_1\)-penalised regression in generalised linear models
- Distance metric learning for graph structured data
- Structure estimation of binary graphical models on stratified data: application to the description of injury tables for victims of road accidents
- A safe screening rule for accelerating weighted twin support vector machine
- Safe feature identification rule for fused Lasso by an extra dual variable
- Fast Lasso-type safe screening for Fine-Gray competing risks model with ultrahigh dimensional covariates
- Conducting sparse feature selection on arbitrarily long phrases in text corpora with a focus on interpretability
- Screening for a reweighted penalized conditional gradient method
- Fast stepwise regression based on multidimensional indexes
- A hybrid acceleration strategy for nonparallel support vector machine
- Optimization in high dimensions via accelerated, parallel, and proximal coordinate descent
- A novel ramp loss-based multi-task twin support vector machine with multi-parameter safe acceleration
- scientific article; zbMATH DE number 7626751 (Why is no real title available?)
- Feature screening strategy for non-convex sparse logistic regression with log sum penalty
- A decomposition method for Lasso problems with zero-sum constraint
- Safe feature elimination for non-negativity constrained convex optimization
- Adaptive sieving: a dimension reduction technique for sparse optimization problems
- Graphical Lasso and thresholding: equivalence and closed-form solutions
- An active-set proximal-Newton algorithm for \(\ell_1\) regularized optimization problems with box constraints
- scientific article; zbMATH DE number 7306914 (Why is no real title available?)
- Scaling up twin support vector regression with safe screening rule
- A safe reinforced feature screening strategy for Lasso based on feasible solutions
- Screening rules and its complexity for active set identification
This page was built for publication: Safe Feature Elimination in Sparse Supervised Learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4906144)
