Feature Selection for Ridge Regression with Provable Guarantees
From MaRDI portal
Abstract: We introduce single-set spectral sparsification as a deterministic sampling based feature selection technique for regularized least squares classification, which is the classification analogue to ridge regression. The method is unsupervised and gives worst-case guarantees of the generalization power of the classification function after feature selection with respect to the classification function obtained using all features. We also introduce leverage-score sampling as an unsupervised randomized feature selection method for ridge regression. We provide risk bounds for both single-set spectral sparsification and leverage-score sampling on ridge regression in the fixed design setting and show that the risk in the sampled space is comparable to the risk in the full-feature space. We perform experiments on synthetic and real-world datasets, namely a subset of TechTC-300 datasets, to support our theory. Experimental results indicate that the proposed methods perform better than the existing feature selection methods.
Recommendations
- Feature selection for linear SVM with provable guarantees
- Revisiting feature selection for linear models with FDR and power guarantees
- On the consistency of feature selection using greedy least squares regression
- Optimal learning in linear regression with combinatorial feature selection
- Efficient learning and feature selection in high-dimensional regression
- Another look at linear programming for feature selection via methods of regularization
- Efficient feature selection using shrinkage estimators
- A semi-parametric approach to feature selection in high-dimensional linear regression models
Cites work
- Deterministic Feature Selection for K-Means Clustering
- Feature selection for linear SVM with provable guarantees
- scientific article; zbMATH DE number 47363 (Why is no real title available?)
- scientific article; zbMATH DE number 1869443 (Why is no real title available?)
- scientific article; zbMATH DE number 2115052 (Why is no real title available?)
- Jacobi’s Method is More Accurate than QR
- Multicategory proximal support vector machine classifiers
- Sampling algorithms for l₂ regression and applications
- Sampling from large matrices
- Second order cone programming formulations for feature selection
- Twice-Ramanujan sparsifiers
Cited in
(3)
This page was built for publication: Feature Selection for Ridge Regression with Provable Guarantees
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380416)
