Priors in Bayesian Deep Learning: A Review
From MaRDI portal
Abstract: While the choice of prior is one of the most critical parts of the Bayesian inference workflow, recent Bayesian deep learning models have often fallen back on vague priors, such as standard Gaussians. In this review, we highlight the importance of prior choices for Bayesian deep learning and present an overview of different priors that have been proposed for (deep) Gaussian processes, variational autoencoders, and Bayesian neural networks. We also outline different methods of learning priors for these models from data. We hope to motivate practitioners in Bayesian deep learning to think more carefully about the prior specification for their models and to provide them with some inspiration in this regard.
Cites work
- A unifying view of sparse approximate Gaussian process regression
- An essay towards solving a problem in the doctrine of chances. By the late Rev. Mr. Bayes, F. R. S. communicated by Mr. Price, in a letter to John Canton, A. M. F. R. S.
- An invariant form for the prior probability in estimation problems
- Attention is Turing-complete
- Deep exploration via randomized value functions
- Default Bayesian analysis with global-local shrinkage priors
- Dutch book against some `objective' priors
- Foundations of data science
- Gaussian processes for machine learning.
- Graphical models, exponential families, and variational inference
- How deep are deep Gaussian processes?
- scientific article; zbMATH DE number 6378127 (Why is no real title available?)
- scientific article; zbMATH DE number 3844820 (Why is no real title available?)
- scientific article; zbMATH DE number 1304257 (Why is no real title available?)
- scientific article; zbMATH DE number 1113195 (Why is no real title available?)
- scientific article; zbMATH DE number 3434895 (Why is no real title available?)
- scientific article; zbMATH DE number 947414 (Why is no real title available?)
- scientific article; zbMATH DE number 854960 (Why is no real title available?)
- scientific article; zbMATH DE number 1448976 (Why is no real title available?)
- scientific article; zbMATH DE number 3426640 (Why is no real title available?)
- scientific article; zbMATH DE number 3063387 (Why is no real title available?)
- Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
- Machine learning. A probabilistic perspective
- Marginal Likelihood Computation for Model Selection and Hypothesis Testing: An Extensive Review
- On advances in statistical modeling of natural images
- On the marginal likelihood and cross-validation
- Prior distributions for variance parameters in hierarchical models (Comment on article by Browne and Draper)
- Prior Probabilities
- The Bayesian Choice
- The Bernstein-von Mises theorem under misspecification
- Wide neural networks of any depth evolve as linear models under gradient descent *
Cited in
(10)- Bayesian distillation of deep learning models
- Solution of physics-based inverse problems using conditional generative adversarial networks with full gradient penalty
- -Stable convergence of heavy-/light-tailed infinitely wide neural networks
- Stochastic automatic differentiation for Monte Carlo processes
- Minimax optimality of deep neural networks on dependent data via PAC-Bayes bounds
- Posterior and variational inference for deep neural networks with heavy-tailed weights
- Posterior concentrations of fully-connected Bayesian neural networks with general priors on the weights
- Deep out-of-distribution uncertainty quantification via weight entropy maximization
- The landscape of unfolding with machine learning
- A primer on Bayesian neural networks: review and debates
This page was built for publication: Priors in Bayesian Deep Learning: A Review
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6067601)
