A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression

DOI10.3150/13-BEJ533MaRDI QIDQ396020zbMATH OpenOpenAlexFDO

Authors Michaël Chichignoud, Johannes Lederer

Publication date 8 August 2014

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1207.4447, https://projecteuclid.org/euclid.bj/1402488950

zbMATH Keywords

adaptationHuber contrastLepski's methodM-estimationminimax estimationnonparametric regressionpointwise estimationrobust estimation

Mathematics Subject Classification ID

Minimax procedures in statistical decision theory (62C20) Nonparametric regression and quantile regression (62G08) Asymptotic properties of nonparametric inference (62G20) Nonparametric robustness (62G35)

Abstract: We introduce a robust and fully adaptive method for pointwise estimation in heteroscedastic regression. We allow for noise and design distributions that are unknown and fulfill very weak assumptions only. In particular, we do not impose moment conditions on the noise distribution. Moreover, we do not require a positive density for the design distribution. In a first step, we study the consistency of locally polynomial M-estimators that consist of a contrast and a kernel. Afterwards, minimax results are established over unidimensional H"older spaces for degenerate design. We then choose the contrast and the kernel that minimize an empirical variance term and demonstrate that the corresponding M-estimator is adaptive with respect to the noise and design distributions and adaptive (Huber) minimax for contamination models. In a second step, we additionally choose a data-driven bandwidth via Lepski's method. This leads to an M-estimator that is adaptive with respect to the noise and design distributions and, additionally, adaptive with respect to the smoothness of an isotropic, multivariate, locally polynomial target function. These results are also extended to anisotropic, locally constant target functions. Our data-driven approach provides, in particular, a level of robustness that adapts to the noise, contamination, and outliers.

Review text

The authors introduce a robust method for pointwise estimation in heteroscedastic regression. The method is adaptive with respect to the noise and design distribution (D-adaptive) and the smoothness of the regression function (S-adaptive). The noise and design distributions considered are unknown and fulfill weak assumptions -- moment conditions on the noise distribution are not imposed and positive density for the design distribution is not required. In a first step, the authors derive general properties of M-estimators such as pointwise risk bounds, including S-minimax results for degenerate designs over unidimensional Hölder spaces. After, they choose the contrast and the kernel of the estimators that minimize an empirical variance term and demonstrate that the corresponding M-estimator is adaptive with respect to the noise and design distributions and adaptive Huber minimax for contamination models. Choosing a data-driven bandwidth via Lepski's method leads to an M-estimator that is adaptive with respect to the noise and design distributions as well as with respect to the smoothness of an isotropic, multivariate, locally polynomial target function. The results are also extended to anisotropic, locally constant target functions. This approach provides also a level of robustness that adapt to the noise, contamination, and outliers.

Recommendations

• Adaptive estimators for nonparametric heteroscedastic regression models
• Adaptive \(M\)-estimation in nonparametric regression
• Spatial adaptation in heteroscedastic regression: propagation approach
• On a robust local estimator for the scale function in heteroscedastic nonparametric regression
• scientific article; zbMATH DE number 947421

Cites work

• A constrained risk inequality with applications to nonparametric functional estimation
• Adaptive maximum likelihood estimators of a location parameter
• Adaptive nonparametric estimation of smooth multivariate functions.
• Asymptotic equivalence and adaptive estimation for robust nonparametric regression
• Confidence bands in density estimation
• Convergence of nonparametric robust algorithms of reconstruction of functions
• Convergence of the optimal M-estimator over a parametric family of M-estimators
• From local kernel to nonlocal multiple-model image denoising
• scientific article; zbMATH DE number 3921745 (Why is no real title available?)
• scientific article; zbMATH DE number 3787846 (Why is no real title available?)
• scientific article; zbMATH DE number 4197203 (Why is no real title available?)
• Introduction to nonparametric estimation
• Minimax and minimax adaptive estimation in multiplicative regression: locally Bayesian approach
• Nonlinear estimation in anisotropic multi-index denoising
• Nonparametric regression estimation based on spatially inhomogeneous data: minimax global convergence rates and adaptivity
• Nonparametric signal estimation when there is incomplete information on the noise distribution
• On adaptive inference and confidence bands
• On pointwise adaptive curve estimation based on inhomogeneous data
• On spatially adaptive estimation of nonparametric regression
• Optimal spatial adaptation to inhomogeneous smoothness: An approach based on kernel estimates with variable bandwidth selectors
• Parameter tuning in pointwise adaptation using a propagation approach
• Propagation-separation approach for local likelihood estimation
• Robust Estimation of a Location Parameter
• Robust reconstruction of functions by the local-approximation method
• Robust regression through the Huber's criterion and adaptive lasso penalty
• Robust Statistics
• Robust Statistics
• Sharp estimation in sup norm with random design
• The Bernstein-Orlicz norm and deviation inequalities
• Universal pointwise selection rule in multivariate function estimation

Cited in

(9)

• Estimation of high-dimensional graphical models using regularized score matching
• Spatial adaptation in heteroscedastic regression: propagation approach
• Learning from MOM's principles: Le Cam's approach
• AdaReg: data adaptive robust estimation in linear regression with application in GTEx gene expressions
• Mean estimation and regression under heavy-tailed distributions: A survey
• Adaptive estimation over anisotropic functional classes via oracle approach
• Bandwidth selection in kernel empirical risk minimization via the gradient
• On pointwise adaptive curve estimation based on inhomogeneous data
• Adaptation to lowest density regions with application to support recovery

This page was built for publication: A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q396020)