Rho-estimators revisited: general theory and applications

DOI10.1214/17-AOS1675MaRDI QIDQ1990601zbMATH OpenFDO

Authors L. Birgé, Yannick Baraud

Publication date 25 October 2018

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1605.05051, https://projecteuclid.org/euclid.aos/1536631290

zbMATH Keywords

density estimationrobust estimationmaximum likelihood estimatorsmetric dimensionstatistical modelsVC-classesregression with random design\(\rho\)-estimation

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Density estimation (62G07) Nonparametric regression and quantile regression (62G08) Nonparametric robustness (62G35) Minimax procedures in statistical decision theory (62C20)

Abstract: Following Baraud, Birg'e and Sart (2017), we pursue our attempt to design a robust universal estimator of the joint ditribution of $n$ independent (but not necessarily i.i.d.) observations for an Hellinger-type loss. Given such observations with an unknown joint distribution $mathbfP$ and a dominated model $mathscrQ$ for $mathbfP$, we build an estimator $widehatmathbfP$ based on $mathscrQ$ and measure its risk by an Hellinger-type distance. When $mathbfP$ does belong to the model, this risk is bounded by some quantity which relies on the local complexity of the model in a vicinity of $mathbfP$. In most situations this bound corresponds to the minimax risk over the model (up to a possible logarithmic factor). When $mathbfP$ does not belong to the model, its risk involves an additional bias term proportional to the distance between $mathbfP$ and $mathscrQ$, whatever the true distribution $mathbfP$. From this point of view, this new version of $ho$-estimators improves upon the previous one described in Baraud, Birg'e and Sart (2017) which required that $mathbfP$ be absolutely continuous with respect to some known reference measure. Further additional improvements have been brought as compared to the former construction. In particular, it provides a very general treatment of the regression framework with random design as well as a computationally tractable procedure for aggregating estimators. We also give some conditions for the Maximum Likelihood Estimator to be a $ho$-estimator. Finally, we consider the situation where the Statistician has at disposal many different models and we build a penalized version of the $ho$-estimator for model selection and adaptation purposes. In the regression setting, this penalized estimator not only allows to estimate the regression function but also the distribution of the errors.

Recommendations

• A new method for estimation and model selection: \(\rho\)-estimation
• Rho-estimators for shape restricted density estimation
• A robust alternative to the maximum likelihood method: \(\rho\)-estimation
• Tests and estimation strategies associated to some loss functions
• Robust Bayes-like estimation: rho-Bayes estimation

Cites work

• scientific article; zbMATH DE number 3605754 (Why is no real title available?)
• scientific article; zbMATH DE number 1420699 (Why is no real title available?)
• A distribution-free theory of nonparametric regression
• A new method for estimation and model selection: \(\rho\)-estimation
• Approximation dans les espaces m�triques et th�orie de l'estimation
• Bounding the expectation of the supremum of an empirical process over a (weak) VC-major class
• Concentration inequalities and asymptotic results for ratio type empirical processes
• Convergence of estimates under dimensionality restrictions
• Convergence of stochastic processes
• Estimating the conditional density by histogram type estimators and model selection
• Local Rademacher complexities and oracle inequalities in risk minimization. (2004 IMS Medallion Lecture). (With discussions and rejoinder)
• Maximum Likelihood: An Introduction
• Minimum contrast estimators on sieves: Exponential bounds and rates of convergence
• Model selection via testing: an alternative to (penalized) maximum likelihood estimators.
• Rho-estimators for shape restricted density estimation
• Robust linear least squares regression
• Weak convergence and empirical processes. With applications to statistics

Cited in

(14)

• Robust Bayes-like estimation: rho-Bayes estimation
• From robust tests to Bayes-like posterior distributions
• A robust alternative to the maximum likelihood method: \(\rho\)-estimation
• Tests and estimation strategies associated to some loss functions
• A new method for estimation and model selection: \(\rho\)-estimation
• Robust estimation of a regression function in exponential families
• Robust tests for model selection
• Robust nonparametric regression based on deep ReLU neural networks
• Estimating a density, a hazard rate, and a transition intensity via the \(\rho\)-estimation method
• Rho-estimators for shape restricted density estimation
• Mean estimation and regression under heavy-tailed distributions: A survey
• Estimating a regression function in exponential families by model selection
• Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
• scientific article; zbMATH DE number 7306864 (Why is no real title available?)

This page was built for publication: Rho-estimators revisited: general theory and applications

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1990601)