On the brittleness of Bayesian inference

DOI10.1137/130938633zbMATH Open1341.62094arXiv1308.6306OpenAlexW3101417746MaRDI QIDQ2808261FDOQ2808261

C. Scovel, Houman Owhadi, T. J. Sullivan

Publication date: 20 May 2016

Published in: SIAM Review (Search for Journal in Brave)

Abstract: With the advent of high-performance computing, Bayesian methods are increasingly popular tools for the quantification of uncertainty throughout science and industry. Since these methods impact the making of sometimes critical decisions in increasingly complicated contexts, the sensitivity of their posterior conclusions with respect to the underlying models and prior beliefs is a pressing question for which there currently exist positive and negative results. We report new results suggesting that, although Bayesian methods are robust when the number of possible outcomes is finite or when only a finite number of marginals of the data-generating distribution are unknown, they could be generically brittle when applied to continuous systems (and their discretizations) with finite information on the data-generating distribution. If closeness is defined in terms of the total variation metric or the matching of a finite system of generalized moments, then (1) two practitioners who use arbitrarily close models and observe the same (possibly arbitrarily large amount of) data may reach opposite conclusions; and (2) any given prior and model can be slightly perturbed to achieve any desired posterior conclusions. The mechanism causing brittlenss/robustness suggests that learning and robustness are antagonistic requirements and raises the question of a missing stability condition for using Bayesian Inference in a continuous world under finite information.

Full work available at URL: https://arxiv.org/abs/1308.6306

Recommendations

• Brittleness of Bayesian inference under finite information in a continuous world
• Qualitative robustness in Bayesian inference
• On the robustness of Bayesian networks to learning from non-conjugate sampling
• scientific article; zbMATH DE number 897945
• Robust Bayesian inference via coarsening

zbMATH Keywords

Bayesian inferencemisspecificationrobustnessuncertainty quantificationBayesian sensitivity analysisoptimal uncertainty quantification

Mathematics Subject Classification ID

Bayesian inference (62F15) Nonparametric robustness (62G35)

Cites Work

• Maximum Likelihood Estimation of Misspecified Models
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• A General Qualitative Definition of Robustness
• Probability, Frequency and Reasonable Expectation
• Robust Statistics
• On the consistency of Bayes estimates
• Statistical decision theory and Bayesian analysis. 2nd ed
• The Nonexistence of Certain Statistical Procedures in Nonparametric Problems
• Misspecification in infinite-dimensional Bayesian statistics
• Nonparametric Bernstein-von Mises theorems in Gaussian white noise
• Title not available (Why is that?)
• On the Bernstein-von Mises phenomenon for nonparametric Bayes procedures
• Limiting Behavior of Posterior Distributions when the Model is Incorrect
• An overview of robust Bayesian analysis. (With discussion)
• On the Bernstein-von Mises theorem with infinite-dimensional parameters
• Inverse problems: a Bayesian perspective
• Title not available (Why is that?)
• Merging of Opinions with Increasing Information
• Title not available (Why is that?)
• Probability Theory
• Title not available (Why is that?)
• On the Asymptotic Behavior of Bayes' Estimates in the Discrete Case
• Local sensitivity diagnostics for Bayesian inference
• Title not available (Why is that?)
• Rates of contraction of posterior distributions based on Gaussian process priors
• The Bernstein-von Mises theorem under misspecification
• Dynamics of Bayesian updating with dependent data and misspecified models
• On the Bernstein-von Mises phenomenon in the Gaussian white noise model
• Title not available (Why is that?)
• On Bayes procedures
• Title not available (Why is that?)
• Qualitative robustness in abstract inference
• Optimal uncertainty quantification
• Title not available (Why is that?)
• Qualitative Robustness in Bayesian Inference
• New axioms for rigorous Bayesian probability
• NON-NORMALITY AND TESTS ON VARIANCES
• Linearization of Bayesian robustness problems
• One-sided inference about functionals of a density
• Brittleness of Bayesian inference under finite information in a continuous world
• Brittleness of Bayesian inference and new Selberg formulas
• A Necessary and Sufficient Condition for the Existence of Consistent Estimates
• Title not available (Why is that?)
• Prior envelopes based on belief functions
• Analyse 1: Utilisation des vocalisations pour différencier trois hardes de phoques
• Title not available (Why is that?)
• The dilation phenomenon in robust Bayesian inference. (With discussion)
• Bayes' Theorem in the 21st Century
• Scaled Boolean algebras

Cited In (21)

• Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients
• Bayesian Numerical Homogenization
• Do ideas have shape? Idea registration as the continuous limit of artificial neural networks
• Hyperparameter estimation in Bayesian MAP estimation: parameterizations and consistency
• Probabilistic learning constrained by realizations using a weak formulation of Fourier transform of probability measures
• Bayesian Parameter Identification in Cahn--Hilliard Models for Biological Growth
• Multigrid with Rough Coefficients and Multiresolution Operator Decomposition from Hierarchical Information Games
• Probabilistic learning inference of boundary value problem with uncertainties based on Kullback-Leibler divergence under implicit constraints
• Data-driven uncertainty quantification for predictive flow and transport modeling using support vector machines
• GAN-Based Priors for Quantifying Uncertainty in Supervised Learning
• Uncertainty Quantification in Graph-Based Classification of High Dimensional Data
• Polynomial-chaos-based conditional statistics for probabilistic learning with heterogeneous data applied to atomic collisions of helium on graphite substrate
• Bayesian Probabilistic Numerical Methods
• Transient anisotropic kernel for probabilistic learning on manifolds
• Rejoinder: ``Probabilistic integration: a role in statistical computation?
• Qualitative Robustness in Bayesian Inference
• Model-free data-driven inference in computational mechanics
• On the local Lipschitz stability of Bayesian inverse problems
• Learning dynamical systems from data: a simple cross-validation perspective. I: Parametric kernel flows
• Uncertainty quantification of the 4th kind; optimal posterior accuracy-uncertainty tradeoff with the minimum enclosing ball
• Probabilistic-learning-based stochastic surrogate model from small incomplete datasets for nonlinear dynamical systems

Uses Software

• BayesDA

This page was built for publication: On the brittleness of Bayesian inference

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2808261)