Mean estimation and regression under heavy-tailed distributions: A survey
From MaRDI portal
Publication:2329044
Abstract: We survey some of the recent advances in mean estimation and regression function estimation. In particular, we describe sub-Gaussian mean estimators for possibly heavy-tailed data both in the univariate and multivariate settings. We focus on estimators based on median-of-means techniques but other methods such as the trimmed mean and Catoni's estimator are also reviewed. We give detailed proofs for the cornerstone results. We dedicate a section on statistical learning problems--in particular, regression function estimation--in the presence of possibly heavy-tailed data.
Recommendations
- Loss minimization and parameter estimation with heavy tails
- Sub-Gaussian mean estimators
- Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
- Mean estimation with sub-Gaussian rates in polynomial time
- Robust sub-Gaussian estimation of a mean vector in nearly linear time
Cites work
- A distribution-free theory of nonparametric regression
- A new method for estimation and model selection: \(\rho\)-estimation
- A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression
- A theory of the learnable
- An optimal (ϵ,δ)‐randomized approximation scheme for the mean of random variables with bounded relative variance
- An unrestricted learning procedure
- Bandits With Heavy Tail
- Challenging the empirical mean and empirical variance: a deviation study
- Characterization of the law of the iterated logarithm in Banach spaces
- Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
- Concentration inequalities. A nonasymptotic theory of independence
- Efficient algorithms and lower bounds for robust linear regression
- Empirical risk minimization for heavy-tailed losses
- Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
- Geometric median and robust estimation in Banach spaces
- Geometric median in nearly linear time
- High-dimensional robust precision matrix estimation: cellwise corruption under \(\epsilon \)-contamination
- scientific article; zbMATH DE number 3954047 (Why is no real title available?)
- scientific article; zbMATH DE number 3790208 (Why is no real title available?)
- scientific article; zbMATH DE number 3541764 (Why is no real title available?)
- scientific article; zbMATH DE number 3560419 (Why is no real title available?)
- scientific article; zbMATH DE number 741240 (Why is no real title available?)
- scientific article; zbMATH DE number 3446442 (Why is no real title available?)
- scientific article; zbMATH DE number 893887 (Why is no real title available?)
- scientific article; zbMATH DE number 1420699 (Why is no real title available?)
- scientific article; zbMATH DE number 3188867 (Why is no real title available?)
- Introduction to nonparametric estimation
- Learnability and the Vapnik-Chervonenkis dimension
- Learning from MOM's principles: Le Cam's approach
- Learning from untrusted data
- Learning without concentration
- Learning without concentration for general loss functions
- Local Rademacher complexities
- Loss minimization and parameter estimation with heavy tails
- Mixture models, robustness, and sum of squares proofs
- Multiscale dictionary learning: non-asymptotic bounds and robustness
- Near-optimal mean estimators with respect to general norms
- Neural Network Learning
- On Some Robust Estimates of Location
- On the absolute constants in the Berry-Esseen-type inequalities
- On the estimation of the mean of a random vector
- Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
- Performance of empirical risk minimization in linear aggregation
- Probability Inequalities for Sums of Bounded Random Variables
- Random generation of combinatorial structures from a uniform distribution
- Regularization, sparse recovery, and median-of-means tournaments
- Rho-estimators revisited: general theory and applications
- Robust classification via MOM minimization
- Robust covariance estimation under \(L_4\)-\(L_2\) norm equivalence
- Robust Estimation of a Location Parameter
- Robust estimators in high-dimensions without the computational intractability
- Robust linear least squares regression
- Robust machine learning by median-of-means: theory and practice
- Robust moment estimation and improved clustering via sum of squares
- Robust multivariate mean estimation: the optimality of trimmed mean
- Robust Statistics
- Robustly learning a Gaussian: getting optimal error, efficiently
- Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries
- Sub-Gaussian estimators of the mean of a random vector
- Sub-Gaussian mean estimators
- The asymptotic distribution of the trimmed mean
- The concentration of measure phenomenon
- The space complexity of approximating the frequency moments
- Weak convergence and empirical processes. With applications to statistics
Cited in
(75)- Challenging the empirical mean and empirical variance: a deviation study
- Nearly optimal robust mean estimation via empirical characteristic function
- Iteratively reweighted _1-penalized robust regression
- K-bMOM: A robust Lloyd-type clustering algorithm based on bootstrap median-of-means
- Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
- A generalized Catoni's M-estimator under finite \(\alpha\)-th moment assumption with \(\alpha \in (1,2)\)
- Robust and efficient mean estimation: an approach based on the properties of self-normalized sums
- On Monte-Carlo methods in convex stochastic optimization
- Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices
- Consistency of invariance-based randomization tests
- Distribution-free robust linear regression
- Robust sub-Gaussian estimation of a mean vector in nearly linear time
- On the measure of anchored Gaussian simplices, with applications to multivariate medians
- Concentration study of M-estimators using the influence function
- Robust machine learning by median-of-means: theory and practice
- Mean estimation with sub-Gaussian rates in polynomial time
- Regularization, sparse recovery, and median-of-means tournaments
- Robust multivariate mean estimation: the optimality of trimmed mean
- Posterior concentration and fast convergence rates for generalized Bayesian learning
- A new method for solving the problem of the mean estimation when the underlying regression function is discontinuous
- Estimating location parameters in sample-heterogeneous distributions
- An overview of heavy-tail extensions of multivariate Gaussian distribution and their relations
- scientific article; zbMATH DE number 7626718 (Why is no real title available?)
- Recovery of Sobolev functions restricted to iid sampling
- Sub-Gaussian mean estimators
- Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
- Learning in repeated auctions
- On the robustness to adversarial corruption and to heavy-tailed data of the Stahel–Donoho median of means
- Non-asymptotic analysis and inference for an outlyingness induced winsorized mean
- Nonasymptotic one- and two-sample tests in high dimension with unknown covariance structure
- Topics in robust statistical learning
- Robustifying Markowitz
- Robust tests for scatter separability beyond Gaussianity
- Catoni-style confidence sequences for heavy-tailed mean estimation
- Robust supervised learning with coordinate gradient descent
- Dimension-free bounds for sums of independent matrices and simple tensors via the variational principle
- Mean estimation in high dimension
- A survey of learning criteria going beyond the usual risk
- The geometric median and applications to robust mean estimation
- Gaussian differentially private robust mean estimation and inference
- Robust subgaussian estimation with VC-dimension
- Exponential concentration for geometric-median-of-means in non-positive curvature spaces
- Robust statistical boosting with quantile-based adaptive loss functions
- Covariance estimation with direction dependence accuracy
- Covariance estimation: optimal dimension-free guarantees for adversarial corruption and heavy tails
- Convergence and concentration properties of constant step-size SGD through Markov chains
- Sharp concentration of simple random tensors
- Accelerated convergence of error quantiles using robust randomized quasi Monte Carlo methods
- Tilted least squares robust estimators
- Trimmed sample means for robust uniform mean estimation and regression
- Uniform bounds for robust mean estimators
- Minimax off-policy evaluation and learning with subgaussian and differentiable importance weighting
- Statistical error bounds for weighted mean and median with application to robust aggregation of cryptocurrency data
- Statistically optimal robust mean and covariance estimation for anisotropic gaussians
- When Heavy Tails Disrupt Statistical Inference
- On deviation probabilities in non-parametric regression with heavy-tailed noise
- Extrapolated Cross-Validation for Randomized Ensembles
- Robust Regression with Covariate Filtering: Heavy Tails and Adversarial Contamination
- Residual permutation test for regression coefficient testing
- Generalized median of means principle for Bayesian inference
- Efficient private SCO for heavy-tailed data via averaged clipping
- Computationally efficient and statistically optimal robust high-dimensional linear regression
- Tight bounds for local Glivenko-Cantelli
- CRIMED: lower and upper bounds on regret for bandits with unbounded stochastic corruption
- Algorithmic stability of heavy-tailed stochastic gradient descent on least squares
- Online robust estimation and bootstrap inference for function-on-scalar regression
- Structure preservation via the Wasserstein distance
- Noisy recovery from random linear observations: sharp minimax rates under elliptical constraints
- High-dimensional robust regression under heavy-tailed data: asymptotics and universality
- Asymptotic dynamics of alternating minimization for bilinear regression
- Uniform mean estimation via generic chaining
- Nonasymptotic heavy-tailed mean estimation in smooth Banach spaces
- Robust gradient boosting for generalized additive models for location, scale and shape
- On the distance between mean and geometric median in high dimensions
- Closed-Loop Generative Selection: Convergence, Memory, and Noisy Oracles
This page was built for publication: Mean estimation and regression under heavy-tailed distributions: A survey
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2329044)
