Robust linear least squares regression
From MaRDI portal
Abstract: We consider the problem of robustly predicting as well as the best linear combination of given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. For the ridge estimator and the ordinary least squares estimator, and their variants, we provide new risk bounds of order without logarithmic factor unlike some standard results, where is the size of the training data. We also provide a new estimator with better deviations in the presence of heavy-tailed noise. It is based on truncating differences of losses in a min--max framework and satisfies a risk bound both in expectation and in deviations. The key common surprising factor of these results is the absence of exponential moment condition on the output distribution while achieving exponential deviations. All risk bounds are obtained through a PAC-Bayesian analysis on truncated differences of losses. Experimental results strongly back up our truncated min--max estimator.
Recommendations
Cites work
- scientific article; zbMATH DE number 3905646 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- A distribution-free theory of nonparametric regression
- Aggregating regression procedures to improve performance
- Challenging the empirical mean and empirical variance: a deviation study
- High breakdown-point and high efficiency robust estimates for regression
- Learning Theory and Kernel Machines
- Minimum contrast estimators on sieves: Exponential bounds and rates of convergence
- Model selection for regression on a fixed design
- Piecewise Polynomial Estimation of a Regression Function
- Robust linear least squares regression
Cited in
(50)- Geometric median and robust estimation in Banach spaces
- A new method for estimation and model selection: \(\rho\)-estimation
- Robust linear least squares regression
- Empirical risk minimization for heavy-tailed losses
- scientific article; zbMATH DE number 7415112 (Why is no real title available?)
- Finite sample performance of linear least squares estimation
- Finite impulse response models: a non-asymptotic analysis of the least squares estimator
- Convergence rates of least squares regression estimators with heavy-tailed errors
- A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
- Estimation from nonlinear observations via convex programming with application to bilinear regression
- Dimension-free bounds for sums of independent matrices and simple tensors via the variational principle
- The lower tail of random quadratic forms with applications to ordinary least squares
- Robust linear regression with broad distributions of errors
- On least squares estimation under heteroscedastic and heavy-tailed errors
- On some recent advances on high dimensional Bayesian statistics
- PAC-Bayesian estimation and prediction in sparse additive models
- Algorithms of robust stochastic optimization based on mirror descent method
- Performance of empirical risk minimization in linear aggregation
- Robust statistical learning with Lipschitz and convex loss functions
- Distribution-free robust linear regression
- Confidence regions and minimax rates in outlier-robust estimation on the probability simplex
- Random design analysis of ridge regression
- Learning from MOM's principles: Le Cam's approach
- ARFIS: an adaptive robust model for regression with heavy-tailed distribution
- Aggregated hold out for sparse linear regression with a robust loss function
- A User-Friendly Computational Framework for Robust Structured Regression with the L2 Criterion
- Regularization, sparse recovery, and median-of-means tournaments
- Suboptimality of constrained least squares and improvements via non-linear predictors
- Robust inference via multiplier bootstrap
- Challenging the empirical mean and empirical variance: a deviation study
- Robust subgaussian estimation with VC-dimension
- Robust regression using biased objectives
- Fast and approximate exhaustive variable selection for generalised linear models with APES
- Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices
- scientific article; zbMATH DE number 7307489 (Why is no real title available?)
- Robust Linear Regression via $\ell_0$ Regularization
- Empirical risk minimization is optimal for the convex aggregation problem
- Mean estimation and regression under heavy-tailed distributions: A survey
- Robust classification via MOM minimization
- Rho-estimators revisited: general theory and applications
- Statistical analysis of the moving least-squares method with unbounded sampling
- A New Principle for Tuning-Free Huber Regression
- scientific article; zbMATH DE number 7370566 (Why is no real title available?)
- Robust machine learning by median-of-means: theory and practice
- User-friendly Introduction to PAC-Bayes Bounds
- On the properties of variational approximations of Gibbs posteriors
- Simpler PAC-Bayesian bounds for hostile data
- Robust lagfactors
- A minimax framework for quantifying risk-fairness trade-off in regression
- Least squares regression under weak moment conditions
This page was built for publication: Robust linear least squares regression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q661182)
