Concentration of tempered posteriors and of their variational approximations
From MaRDI portal
Publication:2196229
Abstract: While Bayesian methods are extremely popular in statistics and machine learning, their application to massive datasets is often challenging, when possible at all. Indeed, the classical MCMC algorithms are prohibitively slow when both the model dimension and the sample size are large. Variational Bayesian methods aim at approximating the posterior by a distribution in a tractable family. Thus, MCMC are replaced by an optimization algorithm which is orders of magnitude faster. VB methods have been applied in such computationally demanding applications as including collaborative filtering, image and video processing, NLP and text processing... However, despite very nice results in practice, the theoretical properties of these approximations are usually not known. In this paper, we propose a general approach to prove the concentration of variational approximations of fractional posteriors. We apply our theory to two examples: matrix completion, and Gaussian VB.
Recommendations
- Frequentist consistency of variational Bayes
- Convergence rates of variational posterior distributions
- \(\alpha\)-variational inference with statistical guarantees
- Consistency of variational Bayes inference for estimation and model selection in mixtures
- On the properties of variational approximations of Gibbs posteriors
Cites work
- scientific article; zbMATH DE number 6377992 (Why is no real title available?)
- scientific article; zbMATH DE number 5544465 (Why is no real title available?)
- 1-bit matrix completion: PAC-Bayesian analysis of a variational approximation
- A Bayesian approach for noisy matrix completion: optimal rate under general sampling distribution
- Active Learning and Basis Selection for Kernel-Based Linear Models: A Bayesian Perspective
- Bayesian fractional posteriors
- Convergence rates of posterior distributions.
- Estimation bounds and sharp oracle inequalities of regularized procedures with Lipschitz loss functions
- Frequentist consistency of variational Bayes
- Fundamentals of nonparametric Bayesian inference
- Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
- Introduction to nonparametric estimation
- Leave Pima Indians alone: binary regression as a benchmark for Bayesian computation
- Noisy low-rank matrix completion with general sampling distribution
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- On the properties of variational approximations of Gibbs posteriors
- Online learning and online convex optimization
- Pattern recognition and machine learning.
- Rényi Divergence and Kullback-Leibler Divergence
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- The Power of Convex Relaxation: Near-Optimal Matrix Completion
- The Variational Gaussian Approximation Revisited
- The safe Bayesian. Learning the learning rate via the mixability gap
Cited in
(55)- Variational Bayesian methods for spatial data analysis
- \(\alpha\)-variational inference with statistical guarantees
- On the convergence of coordinate ascent variational inference
- Robust estimation with variational Bayes in presence of competing risks
- Variational inference for generalized linear mixed models using partially noncentered parametrizations
- Condition for perfect dimensionality recovery by variational Bayesian PCA
- User-friendly Introduction to PAC-Bayes Bounds
- Projective Integral Updates for High-Dimensional Variational Inference
- Concentration of a sparse Bayesian model with horseshoe prior in estimating high-dimensional precision matrix
- Misclassification excess risk bounds for 1-bit matrix completion
- Frequentist consistency of variational Bayes
- Convergence rates of variational posterior distributions
- Loss-Based Variational Bayes Prediction
- Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
- Concentration properties of fractional posterior in 1-bit matrix completion
- scientific article; zbMATH DE number 7306861 (Why is no real title available?)
- Variational Bayesian methods: concepts and neuroimage applications
- Covariances, robustness, and variational Bayes
- Variational Bayes in private settings (VIPS)
- Unified algorithms for RL with decision-estimation coefficients: PAC, reward-free, preference-based learning and beyond
- Tight risk bound for high dimensional time series completion
- Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression
- On the properties of variational approximations of Gibbs posteriors
- Unbiased MLMC-based variational Bayes for likelihood-free inference
- On high-dimensional classification by sparse generalized Bayesian logistic regression
- VAE-KRnet and its applications to variational Bayes
- Comprehensive study of variational Bayes classification for dense deep neural networks
- Kullback-Leibler excess risk bounds for exponential weighted aggregation in generalized linear models
- Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance
- Statistical foundation of variational Bayes neural networks
- Heavy-tailed Bayesian nonparametric adaptation
- Online estimation and community detection of network point processes for event streams
- Adaptive variational Bayes: optimality, computation and applications
- Divergence measures and a general framework for local variational approximation
- On the approximation accuracy of Gaussian variational inference
- Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization
- Consistency of variational Bayes inference for estimation and model selection in mixtures
- Advances in Bayesian model selection consistency for high-dimensional generalized linear models
- Semiparametric posterior corrections
- The computational asymptotics of Gaussian variational inference and the Laplace approximation
- Variational Bayes on manifolds
- On properties of fractional posterior in generalized reduced-rank regression
- Consistency of variational inference for Besov priors in non-linear inverse problems
- Approximate Riemannian conjugate gradient learning for fixed-form variational Bayes
- Posterior and variational inference for deep neural networks with heavy-tailed weights
- Updating variational Bayes: fast sequential posterior inference
- Variational Bayes neural networks for high-dimensional non parametric regression: Minimax optimality and adaptivity
- Numerical comparisons between Bayesian and frequentist low-rank matrix completion: estimation accuracy and uncertainty quantification
- High-dimensional sparse classification using exponential weighting with empirical hinge loss
- On variational Bayes estimation and variational information criteria for linear regression models
- Approximating Bayes in the 21st century
- Emerging directions in Bayesian computation
- Asymptotic consistency of loss-calibrated variational Bayes
- Variational Bayesian approximation. A rigorous approach
- Averaged collapsed variational Bayes inference
This page was built for publication: Concentration of tempered posteriors and of their variational approximations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2196229)
