Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
From MaRDI portal
Abstract: We empirically show that Bayesian inference can be inconsistent under misspecification in simple linear regression problems, both in a model averaging/selection and in a Bayesian ridge regression setting. We use the standard linear model, which assumes homoskedasticity, whereas the data are heteroskedastic, and observe that the posterior puts its mass on ever more high-dimensional models as the sample size increases. To remedy the problem, we equip the likelihood in Bayes' theorem with an exponent called the learning rate, and we propose the Safe Bayesian method to learn the learning rate from the data. SafeBayes tends to select small learning rates as soon the standard posterior is not `cumulatively concentrated', and its results on our data are quite encouraging.
Recommendations
Cited in
(92)- Robust approximate Bayesian inference
- Bayesian fractional posteriors
- On Bayesian oracle properties
- Consistency of variational Bayes inference for estimation and model selection in mixtures
- Approximate models and robust decisions
- Contextuality of misspecification and data-dependent losses
- Rejoinder: Approximate models and robust decisions
- The Bayes estimator in a misspecified linear regression model
- Decision-based model selection
- Inference without compatibility: using exponential weighting for inference on a parameter of a linear model
- Price probabilities: a class of Bayesian and non-Bayesian prediction rules
- Gibbs posterior inference on multivariate quantiles
- Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
- Gibbs posterior convergence and the thermodynamic formalism
- Comments on: ``On active learning methods for manifold data
- Model-free posterior inference on the area under the receiver operating characteristic curve
- Concentration of tempered posteriors and of their variational approximations
- Empirical priors and coverage of posterior credible sets in a sparse normal mean model
- AdaReg: data adaptive robust estimation in linear regression with application in GTEx gene expressions
- Minimax posterior convergence rates and model selection consistency in high-dimensional DAG models based on sparse Cholesky factors
- Suboptimal behavior of Bayes and MDL in classification under misspecification
- Posterior concentration and fast convergence rates for generalized Bayesian learning
- Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons
- Gibbs posterior concentration rates under sub-exponential type losses
- Tractable Bayesian variable selection: beyond normality
- The safe Bayesian. Learning the learning rate via the mixability gap
- Bayesian inference with misspecified models
- Learning Theory
- Fast rates for general unbounded loss functions: from ERM to generalized Bayes
- Empirical priors for prediction in sparse high-dimensional linear regression
- Minimum description length revisited
- Debiasing inference with approximate covariance matrices and other unidentified biases
- Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
- General Robust Bayes Pseudo-Posteriors: Exponential Convergence Results with Applications
- Universal inference
- Model Misspecification in Approximate Bayesian Computation: Consequences and Diagnostics
- Model misspecification, Bayesian versus credibility estimation, and Gibbs posteriors
- Gibbs posterior inference on value-at-risk
- Generalized Bayes approach to inverse problems with model misspecification
- Priors in Bayesian Deep Learning: A Review
- A reduced-rank approach to predicting multiple binary responses through machine learning
- Nonparametric Posterior Learning for Emission Tomography
- Generalized Bayes Quantification Learning under Dataset Shift
- Bernstein-von Mises theorem and misspecified models: a review
- A comparison of learning rate selection methods in generalized Bayesian inference
- User-friendly Introduction to PAC-Bayes Bounds
- Modularized Bayesian analyses and cutting feedback in likelihood-free inference
- Confidence and discoveries with e-values
- Joint production in stochastic non-parametric envelopment of data with firm-specific directions
- An efficient adaptive MCMC algorithm for pseudo-Bayesian quantum tomography
- The no-free-lunch theorems of supervised learning
- Bayesian decision-theoretic design of experiments under an alternative model
- The importance Markov chain
- Modern Bayesian experimental design
- Sampling algorithms in statistical physics: a guide for statistics and machine learning
- Empirical Bayes inference in sparse high-dimensional generalized linear models
- Optimal experimental design: formulations and computations
- A nonparametric Bayesian method for dose finding in drug combinations cancer trials
- High-dimensional sparse classification using exponential weighting with empirical hinge loss
- Empirical priors and posterior concentration in a piecewise polynomial sequence model
- Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance
- Safe Bayes for single index quantile regression estimation and variable selection
- Sampling from density power divergence-based generalized posterior distribution via stochastic optimization
- Rational beliefs when the truth is not an option
- Cutting feedback and modularized analyses in generalized Bayesian inference
- On high-dimensional classification by sparse generalized Bayesian logistic regression
- Bayesian experimental design for model discrepancy calibration: an auto-differentiable ensemble Kalman inversion approach
- Multiple testing in generalized universal inference
- Advances in Bayesian model selection consistency for high-dimensional generalized linear models
- Generalized Bayesian inference for causal effects using the covariate balancing procedure
- Combining experimental and observational data through a power likelihood
- High-dimensional prediction for count response via sparse exponential weights
- Bayesian inference for generalized linear models via quasi-posteriors
- Active learning of model discrepancy with Bayesian experimental design
- On properties of fractional posterior in generalized reduced-rank regression
- Predictive performance of power posteriors
- Generalized Bayesian method for diagnostic classification models
- Concentration of a sparse Bayesian model with horseshoe prior in estimating high-dimensional precision matrix
- Synthetic Likelihood in Misspecified Models
- Laplace and saddlepoint approximations in high dimensions
- A sparse PAC-Bayesian approach for high-dimensional quantile prediction
- Concentration properties of fractional posterior in 1-bit matrix completion
- Generalized geographically weighted regression model within a modularized Bayesian framework
- Uncertainty quantification using martingales for misspecified Gaussian processes
- Composite dyadic models for spatio-temporal data
- Sequential Gibbs posteriors with applications to principal component analysis
- Bayesian surrogate training on multiple data sources: a hybrid modeling strategy
- Robust probabilistic inference via a constrained transport metric (with discussion)
- Under-reaction: irrational behavior or robust response to model misspecification?
- Generalized universal inference on risk minimizers
- Posterior Risk of Modular and Semi-Modular Bayesian Inference
- When posterior predictive checks cannot identify learning rates
This page was built for publication: Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1699707)
