Learning from MOM's principles: Le Cam's approach

DOI10.1016/J.SPA.2018.11.024MaRDI QIDQ2010482zbMATH OpenOpenAlexWikidataFDO

Authors Guillaume Lecué, M. Lerasle

Publication date 27 November 2019

Published in Stochastic Processes and their Applications (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1701.01961

zbMATH Keywords

robust statisticsstatistical learninghigh dimensional statistics

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Nonparametric robustness (62G35) Gaussian processes (60G15)

Abstract: We obtain estimation error rates for estimators obtained by aggregation of regularized median-of-means tests, following a construction of Le Cam. The results hold with exponentially large probability -- as in the gaussian framework with independent noise- under only weak moments assumptions on data and without assuming independence between noise and design. Any norm may be used for regularization. When it has some sparsity inducing power we recover sparse rates of convergence. The procedure is robust since a large part of data may be corrupted, these outliers have nothing to do with the oracle we want to reconstruct. Our general risk bound is of order �egin{equation*} maxleft(mbox{minimax rate in the i.i.d. setup}, frac{ ext{number of outliers}}{ ext{number of observations}} ight) enspace. end{equation*}In particular, the number of outliers may be as large as (number of data) $imes$(minimax rate) without affecting this rate. The other data do not have to be identically distributed but should only have equivalent $L^1$ and $L^2$ moments. For example, the minimax rate $slog(ed/s)/N$ of recovery of a $s$-sparse vector in $mathbb{R}^d$ is achieved with exponentially large probability by a median-of-means version of the LASSO when the noise has $q_0$ moments for some $q_0>2$, the entries of the design matrix should have $C_{0}log(ed)$ moments and the dataset can be corrupted up to $C_{1}slog(ed/s)$ outliers.

Recommendations

• Robust machine learning by median-of-means: theory and practice
• Robust statistical learning with Lipschitz and convex loss functions
• On convergence rates of convex regression in multiple dimensions
• Learning without concentration for general loss functions
• Robust classification via MOM minimization

Cites work

• scientific article; zbMATH DE number 6388313 (Why is no real title available?)
• scientific article; zbMATH DE number 4170917 (Why is no real title available?)
• scientific article; zbMATH DE number 3790208 (Why is no real title available?)
• scientific article; zbMATH DE number 28602 (Why is no real title available?)
• scientific article; zbMATH DE number 49190 (Why is no real title available?)
• scientific article; zbMATH DE number 1332320 (Why is no real title available?)
• A new method for estimation and model selection: \(\rho\)-estimation
• A remark on the diameter of random sections of convex bodies
• A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression
• Asymptotic methods in statistical decision theory
• Bounding the smallest singular value of a random matrix without concentration
• Challenging the empirical mean and empirical variance: a deviation study
• Concentration inequalities. A nonasymptotic theory of independence
• Convergence of estimates under dimensionality restrictions
• Empirical risk minimization for heavy-tailed losses
• Estimating the intensity of a random measure by histogram type estimators
• Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
• Estimation of the transition density of a Markov chain
• Estimator selection in the Gaussian setting
• Estimator selection with respect to Hellinger-type risks
• Learning without concentration
• Model selection via testing: an alternative to (penalized) maximum likelihood estimators.
• On aggregation for heavy-tailed classes
• Random generation of combinatorial structures from a uniform distribution
• Rho-estimators for shape restricted density estimation
• Risk minimization by median-of-means tournaments
• Robust Estimation of a Location Parameter
• Robust Statistics
• Robust linear least squares regression
• Robust tests for model selection
• SLOPE is adaptive to unknown sparsity and asymptotically minimax
• SLOPE-adaptive variable selection via convex optimization
• Small ball probabilities for linear images of high-dimensional distributions
• Sparse recovery under weak moment assumptions
• Stabilité et instabilité du risque minimax pour des variables indépendantes équidistribuées
• Statistics for high-dimensional data. Methods, theory and applications.
• Sub-Gaussian mean estimators
• The space complexity of approximating the frequency moments
• Weak convergence and empirical processes. With applications to statistics

Cited in

(21)

• Efficient learning with robust gradient descent
• All-in-one robust estimator of the Gaussian mean
• On the robustness to adversarial corruption and to heavy-tailed data of the Stahel–Donoho median of means
• Robust subgaussian estimation with VC-dimension
• Confidence regions and minimax rates in outlier-robust estimation on the probability simplex
• Robust machine learning by median-of-means: theory and practice
• A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
• Iteratively reweighted \(\ell_1\)-penalized robust regression
• Optimal robust mean and location estimation via convex programs with respect to any pseudo-norms
• Robust statistical learning with Lipschitz and convex loss functions
• Mean estimation and regression under heavy-tailed distributions: A survey
• Regularization, sparse recovery, and median-of-means tournaments
• Core-elements for large-scale least squares estimation
• Robust classification via MOM minimization
• Learning under \((1 + \epsilon)\)-moment conditions
• Multidimensional linear functional estimation in sparse Gaussian models and robust estimation of the mean
• K-bMOM: A robust Lloyd-type clustering algorithm based on bootstrap median-of-means
• Topics in robust statistical learning
• Risk minimization by median-of-means tournaments
• scientific article; zbMATH DE number 7306878 (Why is no real title available?)
• Robust sub-Gaussian estimation of a mean vector in nearly linear time

This page was built for publication: Learning from MOM's principles: Le Cam's approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2010482)