Learning from MOM's principles: Le Cam's approach

DOI10.1016/J.SPA.2018.11.024zbMATH Open1435.62175arXiv1701.01961OpenAlexW2738598838WikidataQ128702491 ScholiaQ128702491MaRDI QIDQ2010482FDOQ2010482

Guillaume Lecué, M. Lerasle

Publication date: 27 November 2019

Published in: Stochastic Processes and their Applications (Search for Journal in Brave)

Abstract: We obtain estimation error rates for estimators obtained by aggregation of regularized median-of-means tests, following a construction of Le Cam. The results hold with exponentially large probability -- as in the gaussian framework with independent noise- under only weak moments assumptions on data and without assuming independence between noise and design. Any norm may be used for regularization. When it has some sparsity inducing power we recover sparse rates of convergence. The procedure is robust since a large part of data may be corrupted, these outliers have nothing to do with the oracle we want to reconstruct. Our general risk bound is of order �egin{equation*} maxleft(mbox{minimax rate in the i.i.d. setup}, frac{ ext{number of outliers}}{ ext{number of observations}} ight) enspace. end{equation*}In particular, the number of outliers may be as large as (number of data) $imes$(minimax rate) without affecting this rate. The other data do not have to be identically distributed but should only have equivalent $L^1$ and $L^2$ moments. For example, the minimax rate $slog(ed/s)/N$ of recovery of a $s$-sparse vector in $mathbb{R}^d$ is achieved with exponentially large probability by a median-of-means version of the LASSO when the noise has $q_0$ moments for some $q_0>2$, the entries of the design matrix should have $C_{0}log(ed)$ moments and the dataset can be corrupted up to $C_{1}slog(ed/s)$ outliers.

Full work available at URL: https://arxiv.org/abs/1701.01961

Recommendations

• Robust machine learning by median-of-means: theory and practice
• Robust statistical learning with Lipschitz and convex loss functions
• On convergence rates of convex regression in multiple dimensions
• Learning without concentration for general loss functions
• Robust classification via MOM minimization

zbMATH Keywords

robust statisticsstatistical learninghigh dimensional statistics

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Nonparametric robustness (62G35) Gaussian processes (60G15)

Cites Work

• Estimator selection in the Gaussian setting
• Weak convergence and empirical processes. With applications to statistics
• Statistics for high-dimensional data. Methods, theory and applications.
• Robust Estimation of a Location Parameter
• Title not available (Why is that?)
• Title not available (Why is that?)
• Robust Statistics
• Asymptotic methods in statistical decision theory
• Challenging the empirical mean and empirical variance: a deviation study
• Convergence of estimates under dimensionality restrictions
• Title not available (Why is that?)
• Title not available (Why is that?)
• Robust linear least squares regression
• The space complexity of approximating the frequency moments
• Model selection via testing: an alternative to (penalized) maximum likelihood estimators.
• Title not available (Why is that?)
• A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression
• Title not available (Why is that?)
• Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
• Estimator selection with respect to Hellinger-type risks
• SLOPE-adaptive variable selection via convex optimization
• Estimating the intensity of a random measure by histogram type estimators
• Random generation of combinatorial structures from a uniform distribution
• Sparse recovery under weak moment assumptions
• Learning without concentration
• Bounding the Smallest Singular Value of a Random Matrix Without Concentration
• Estimation of the transition density of a Markov chain
• A new method for estimation and model selection: \(\rho\)-estimation
• A Remark on the Diameter of Random Sections of Convex Bodies
• Robust tests for model selection
• SLOPE is adaptive to unknown sparsity and asymptotically minimax
• Small Ball Probabilities for Linear Images of High-Dimensional Distributions
• Rho-estimators for shape restricted density estimation
• Stabilité et instabilité du risque minimax pour des variables indépendantes équidistribuées
• Sub-Gaussian mean estimators
• Title not available (Why is that?)
• Empirical risk minimization for heavy-tailed losses
• On aggregation for heavy-tailed classes
• Risk minimization by median-of-means tournaments

Cited In (19)

• Efficient learning with robust gradient descent
• Robust subgaussian estimation with VC-dimension
• All-in-one robust estimator of the Gaussian mean
• Confidence regions and minimax rates in outlier-robust estimation on the probability simplex
• Robust machine learning by median-of-means: theory and practice
• A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
• Iteratively reweighted \(\ell_1\)-penalized robust regression
• Optimal robust mean and location estimation via convex programs with respect to any pseudo-norms
• Robust statistical learning with Lipschitz and convex loss functions
• Mean estimation and regression under heavy-tailed distributions: A survey
• Regularization, sparse recovery, and median-of-means tournaments
• Core-elements for large-scale least squares estimation
• Robust classification via MOM minimization
• Learning under \((1 + \epsilon)\)-moment conditions
• Multidimensional linear functional estimation in sparse Gaussian models and robust estimation of the mean
• K-bMOM: A robust Lloyd-type clustering algorithm based on bootstrap median-of-means
• Topics in robust statistical learning
• Title not available (Why is that?)
• Robust sub-Gaussian estimation of a mean vector in nearly linear time

This page was built for publication: Learning from MOM's principles: Le Cam's approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2010482)