Robust Bayes-like estimation: rho-Bayes estimation

DOI10.1214/20-AOS1948MaRDI QIDQ1996790zbMATH OpenOpenAlexFDO

Authors L. Birgé, Yannick Baraud

Publication date 26 February 2021

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1711.08328, https://projecteuclid.org/euclid.aos/1607677252

zbMATH Keywords

Bayesian estimationdensity estimationmetric dimensionrho-Bayes estimationrobust estimationstatistical modelsVC-classes

Mathematics Subject Classification ID

Minimax procedures in statistical decision theory (62C20) Bayesian inference (62F15) Robustness and adaptive procedures (parametric inference) (62F35)

Abstract: We consider the problem of estimating the joint distribution $P$ of $n$ independent random variables within the Bayes paradigm from a non-asymptotic point of view. Assuming that $P$ admits some density $s$ with respect to a given reference measure, we consider a density model $overlineS$ for $s$ that we endow with a prior distribution $pi$ (with support $overlineS$) and we build a robust alternative to the classical Bayes posterior distribution which possesses similar concentration properties around $s$ whenever it belongs to the model $overlineS$. Furthermore, in density estimation, the Hellinger distance between the classical and the robust posterior distributions tends to 0, as the number of observations tends to infinity, under suitable assumptions on the model and the prior, provided that the model $overlineS$ contains the true density $s$. However, unlike what happens with the classical Bayes posterior distribution, we show that the concentration properties of this new posterior distribution are still preserved in the case of a misspecification of the model, that is when $s$ does not belong to $overlineS$ but is close enough to it with respect to the Hellinger distance.

Review text

The authors [Stochastic Processes Appl. 126, No. 12, 3888--3912 (2016; Zbl 1419.62070); Ann. Stat. 46, No. 6B, 3767--3804 (2018; Zbl 1407.62169)] presented a new alternative to maximum likelihood estimation (MLE), the \(\rho\)-estimation, which leads to robust estimators. The lack of robustness is not specific to the MLE but has also been noticed in the Bayesian framework, since the posterior distribution is not robust with respect to misspecifications of the model. This paper addresses this last problem and proposes, by extending the \(\rho\)-estimation theory, a robust alternative to the classical Bayes posterior distribution. It is proved that this new alternative posterior distribution possesses concentration properties under no misspecification similar to the ones of the classical posterior distribution, In this last case, under some suitable assumptions imposed to the model and the prior, its Hellinger distance from the classical posterior distribution tends to zero, as the number of observations tends to infinity. Moreover, it is shown that the \(\rho\)-posterior possesses robustness properties when the model is misspecified or the data are not i.i.d. provided that the marginal densities are close enough in terms of Hellinger distance to the model considered. Summarizing, it is a very well written paper with an important contribution to the Bayesian framework, since it presents an alternative robust Bayes-like estimation procedure, which can be applied in various statistical frameworks, while at the same time its properties distinguish it from the classical Bayesian procedures.

Recommendations

• scientific article; zbMATH DE number 718741
• Selection of KL neighbourhood in robust Bayesian inference
• A note on the posterior mean robustness for location parameters
• General Robust Bayes Pseudo-Posteriors: Exponential Convergence Results with Applications
• Consistent and robust Bayes procedures for location based on partial information
• scientific article; zbMATH DE number 706362

Cites work

• A General Framework for Updating Belief Distributions
• A new method for estimation and model selection: \(\rho\)-estimation
• A robust alternative to the maximum likelihood method: -estimation
• An MCMC approach to classical estimation.
• Approximation dans les espaces m�triques et th�orie de l'estimation
• Asymptotic methods in statistical decision theory
• Bayesian fractional posteriors
• Bounding the expectation of the supremum of an empirical process over a (weak) VC-major class
• Concentration inequalities and model selection. Ecole d'Eté de Probabilités de Saint-Flour XXXIII -- 2003.
• Convergence of estimates under dimensionality restrictions
• Convergence rates of posterior distributions.
• Finite sample Bernstein-von Mises theorem for semiparametric problems
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• scientific article; zbMATH DE number 5544465 (Why is no real title available?)
• scientific article; zbMATH DE number 3605754 (Why is no real title available?)
• scientific article; zbMATH DE number 1420699 (Why is no real title available?)
• Misspecification in infinite-dimensional Bayesian statistics
• Model selection via testing: an alternative to (penalized) maximum likelihood estimators.
• On estimating a density using Hellinger distance and some other strange facts
• On the contraction properties of some high-dimensional quasi-posterior distributions
• Rho-estimators for shape restricted density estimation
• Rho-estimators revisited: general theory and applications
• Stabilité et instabilité du risque minimax pour des variables indépendantes équidistribuées
• Statistical estimation with model selection
• The Bernstein-von Mises theorem under misspecification
• Weak convergence and empirical processes. With applications to statistics

Cited in

(7)

• Rho-estimators revisited: general theory and applications
• Robust Bayes estimation using the density power divergence
• scientific article; zbMATH DE number 718741 (Why is no real title available?)
• A robust alternative to the maximum likelihood method: -estimation
• Bernstein-von Mises theorem and misspecified models: a review
• From robust tests to Bayes-like posterior distributions
• Generalized median of means principle for Bayesian inference

This page was built for publication: Robust Bayes-like estimation: rho-Bayes estimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1996790)