Concentration of tempered posteriors and of their variational approximations
From MaRDI portal
Publication:2196229
Abstract: While Bayesian methods are extremely popular in statistics and machine learning, their application to massive datasets is often challenging, when possible at all. Indeed, the classical MCMC algorithms are prohibitively slow when both the model dimension and the sample size are large. Variational Bayesian methods aim at approximating the posterior by a distribution in a tractable family. Thus, MCMC are replaced by an optimization algorithm which is orders of magnitude faster. VB methods have been applied in such computationally demanding applications as including collaborative filtering, image and video processing, NLP and text processing... However, despite very nice results in practice, the theoretical properties of these approximations are usually not known. In this paper, we propose a general approach to prove the concentration of variational approximations of fractional posteriors. We apply our theory to two examples: matrix completion, and Gaussian VB.
Recommendations
- Frequentist consistency of variational Bayes
- Convergence rates of variational posterior distributions
- \(\alpha\)-variational inference with statistical guarantees
- Consistency of variational Bayes inference for estimation and model selection in mixtures
- On the properties of variational approximations of Gibbs posteriors
Cites work
- scientific article; zbMATH DE number 6377992 (Why is no real title available?)
- scientific article; zbMATH DE number 5544465 (Why is no real title available?)
- 1-bit matrix completion: PAC-Bayesian analysis of a variational approximation
- A Bayesian approach for noisy matrix completion: optimal rate under general sampling distribution
- Active Learning and Basis Selection for Kernel-Based Linear Models: A Bayesian Perspective
- Bayesian fractional posteriors
- Convergence rates of posterior distributions.
- Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions
- Frequentist consistency of variational Bayes
- Fundamentals of nonparametric Bayesian inference
- Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
- Introduction to nonparametric estimation
- Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation
- Noisy low-rank matrix completion with general sampling distribution
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- On the properties of variational approximations of Gibbs posteriors
- Online learning and online convex optimization
- Pattern recognition and machine learning.
- Rényi Divergence and Kullback-Leibler Divergence
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- The Power of Convex Relaxation: Near-Optimal Matrix Completion
- The Variational Gaussian Approximation Revisited
- The safe Bayesian. Learning the learning rate via the mixability gap
Cited in
(40)- Variational Bayesian methods for spatial data analysis
- \(\alpha\)-variational inference with statistical guarantees
- Robust estimation with variational Bayes in presence of competing risks
- Variational inference for generalized linear mixed models using partially noncentered parametrizations
- Condition for perfect dimensionality recovery by variational Bayesian PCA
- User-friendly Introduction to PAC-Bayes Bounds
- Projective Integral Updates for High-Dimensional Variational Inference
- Frequentist consistency of variational Bayes
- Convergence rates of variational posterior distributions
- Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
- scientific article; zbMATH DE number 7306861 (Why is no real title available?)
- Variational Bayesian methods: concepts and neuroimage applications
- Covariances, robustness, and variational Bayes
- Variational Bayes in private settings (VIPS)
- Tight risk bound for high dimensional time series completion
- Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression
- On the properties of variational approximations of Gibbs posteriors
- Unbiased MLMC-based variational Bayes for likelihood-free inference
- VAE-KRnet and its applications to variational Bayes
- Comprehensive study of variational Bayes classification for dense deep neural networks
- Statistical foundation of variational Bayes neural networks
- Heavy-tailed Bayesian nonparametric adaptation
- Online estimation and community detection of network point processes for event streams
- Adaptive variational Bayes: optimality, computation and applications
- Divergence measures and a general framework for local variational approximation
- On the approximation accuracy of Gaussian variational inference
- Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization
- Consistency of variational Bayes inference for estimation and model selection in mixtures
- The computational asymptotics of Gaussian variational inference and the Laplace approximation
- Variational Bayes on manifolds
- Approximate Riemannian conjugate gradient learning for fixed-form variational Bayes
- Updating variational Bayes: fast sequential posterior inference
- Numerical comparisons between Bayesian and frequentist low-rank matrix completion: estimation accuracy and uncertainty quantification
- High-dimensional sparse classification using exponential weighting with empirical hinge loss
- On variational Bayes estimation and variational information criteria for linear regression models
- Approximating Bayes in the 21st century
- Emerging directions in Bayesian computation
- Asymptotic consistency of loss-calibrated variational Bayes
- Variational Bayesian approximation. A rigorous approach
- Averaged collapsed variational Bayes inference
This page was built for publication: Concentration of tempered posteriors and of their variational approximations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2196229)
