Risk minimization by median-of-means tournaments
From MaRDI portal
Publication:2302852
Prediction theory (aspects of stochastic processes) (60G25) Nonparametric regression and quantile regression (62G08) Statistics of extreme values; tail inference (62G32) General nonlinear regression (62J02) Linear regression; mixed models (62J05) Paired and multiple comparisons; multiple testing (62J15)
Abstract: We consider the classical statistical learning/regression problem, when the value of a real random variable Y is to be predicted based on the observation of another random variable X. Given a class of functions F and a sample of independent copies of (X, Y ), one needs to choose a function f from F such that f(X) approximates Y as well as possible, in the mean-squared sense. We introduce a new procedure, the so-called median-of-means tournament, that achieves the optimal tradeoff between accuracy and confidence under minimal assumptions, and in particular outperforms classical methods based on empirical risk minimization.
This paper develops an approach for squared error minimization based on a median of means estimator and pairwise comparisons of candidate functions. Optimality in terms of accuracy and confidence is discussed in different scenarios, determined in terms of classes of estimation functions and hypotheses on the tails of the distributions. Comparisons with the approach based on empirical risk minimization are performed. Examples of application to linear regression are presented.
Recommendations
Cited in
(50)- Robust regression using biased objectives
- Solvable integration problems and optimal sample size selection
- Learning from MOM's principles: Le Cam's approach
- A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
- Iteratively reweighted _1-penalized robust regression
- K-bMOM: A robust Lloyd-type clustering algorithm based on bootstrap median-of-means
- The smallest singular value of heavy-tailed not necessarily i.i.d. random matrices via random rounding
- Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
- On Monte-Carlo methods in convex stochastic optimization
- Suboptimality of constrained least squares and improvements via non-linear predictors
- Distribution-free robust linear regression
- On least squares estimation under heteroscedastic and heavy-tailed errors
- All-in-one robust estimator of the Gaussian mean
- Optimal robust mean and location estimation via convex programs with respect to any pseudo-norms
- Robust statistical learning with Lipschitz and convex loss functions
- Robust machine learning by median-of-means: theory and practice
- Robust classification via MOM minimization
- Algorithms of robust stochastic optimization based on mirror descent method
- The breakdown point of the median of means tournament
- Regularization, sparse recovery, and median-of-means tournaments
- Faster Hoeffding racing: Bernstein races via jackknife estimates
- Error-correcting tournaments
- Parallel and robust empirical risk minimization via the median trick
- scientific article; zbMATH DE number 7370566 (Why is no real title available?)
- A New Principle for Tuning-Free Huber Regression
- scientific article; zbMATH DE number 7625158 (Why is no real title available?)
- Extending the scope of the small-ball method
- Distributed high-dimensional regression under a quantile loss function
- Median loss decision theory
- Topics in robust statistical learning
- Stable recovery and the coordinate small-ball behaviour of random vectors
- Dualizing Le Cam's method for functional estimation. I: General theory
- Covariance estimation: optimal dimension-free guarantees for adversarial corruption and heavy tails
- Sparse linear regression when noises and covariates are heavy-tailed and contaminated by outliers
- Accelerated convergence of error quantiles using robust randomized quasi Monte Carlo methods
- Robust and computationally efficient gradient-based estimation
- Trimmed sample means for robust uniform mean estimation and regression
- Uniform bounds for robust mean estimators
- On regression in extreme regions
- Outlier robust and sparse estimation of linear regression coefficients
- On deviation probabilities in non-parametric regression with heavy-tailed noise
- Robust Regression with Covariate Filtering: Heavy Tails and Adversarial Contamination
- Large-dimensional independent component analysis: statistical optimality and computational tractability
- Generalized median of means principle for Bayesian inference
- Uniform inference in linear error-in-variables models: Divide-and-conquer
- Computationally efficient and statistically optimal robust high-dimensional linear regression
- Uniform mean estimation via generic chaining
- Robust universal inference for misspecified models
- Robust local Wasserstein Fréchet regression under admissible transport deformations
- Batched bandits with heavy-tailed rewards
This page was built for publication: Risk minimization by median-of-means tournaments
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2302852)
