Iterative feature selection in least square regression estimation
From MaRDI portal
Abstract: In this paper, we focus on regression estimation in both the inductive and the transductive case. We assume that we are given a set of features (which can be a base of functions, but not necessarily). We begin by giving a deviation inequality on the risk of an estimator in every model defined by using a single feature. These models are too simple to be useful by themselves, but we then show how this result motivates an iterative algorithm that performs feature selection in order to build a suitable estimator. We prove that every selected feature actually improves the performance of the estimator. We give all the estimators and results at first in the inductive case, which requires the knowledge of the distribution of the design, and then in the transductive case, in which we do not need to know this distribution.
Recommendations
- Lasso, iterative feature selection and the correlation selector: oracle inequalities and numerical performances
- An algorithm for iterative selection of blocks of features
- Least angle regression. (With discussion)
- Model selection for regularized least-squares algorithm in learning theory
- Least square regression with \(l^{p}\)-coefficient regularization
Cites work
- Aggregated estimators and empirical complexity for least square regression
- Aggregating regression procedures to improve performance
- An introduction to support vector machines and other kernel-based learning methods.
- Approximation and learning by greedy algorithms
- Boosting the margin: a new explanation for the effectiveness of voting methods
- Density estimation by wavelet thresholding
- scientific article; zbMATH DE number 1332320 (Why is no real title available?)
- Ideal spatial adaptation by wavelet shrinkage
- Optimal aggregation of classifiers in statistical learning.
- PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification
- Recursive aggregation of estimators by the mirror descent algorithm with averaging
- Regression in random design and warped wavelets
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- Symmetrization approach to concentration inequalities for empirical processes.
- Topics in non-parametric statistics
- Wavelets, approximation, and statistical applications
Cited in
(8)- Lasso, iterative feature selection and the correlation selector: oracle inequalities and numerical performances
- Fast learning rates in statistical inference through aggregation
- An eager regression method based on best feature projections
- Iterative selection using orthogonal regression techniques
- Inductive confidence machines for regression
- An algorithm for iterative selection of blocks of features
- Density estimation with quadratic loss: a confidence intervals method
- Tight conditions for consistency of variable selection in the context of high dimensionality
This page was built for publication: Iterative feature selection in least square regression estimation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q731450)