Convergence rates of variational posterior distributions
From MaRDI portal
Publication:2215731
DOI10.1214/19-AOS1883zbMATH Open1471.62243arXiv1712.02519OpenAlexW3049347563MaRDI QIDQ2215731FDOQ2215731
Authors: Fengshuo Zhang, Chao Gao 
Publication date: 14 December 2020
Published in: The Annals of Statistics (Search for Journal in Brave)
Abstract: We study convergence rates of variational posterior distributions for nonparametric and high-dimensional inference. We formulate general conditions on prior, likelihood, and variational class that characterize the convergence rates. Under similar "prior mass and testing" conditions considered in the literature, the rate is found to be the sum of two terms. The first term stands for the convergence rate of the true posterior distribution, and the second term is contributed by the variational approximation error. For a class of priors that admit the structure of a mixture of product measures, we propose a novel prior mass condition, under which the variational approximation error of the mean-field class is dominated by convergence rate of the true posterior. We demonstrate the applicability of our general results for various models, prior distributions and variational classes by deriving convergence rates of the corresponding variational posteriors.
Full work available at URL: https://arxiv.org/abs/1712.02519
Recommendations
- Concentration of tempered posteriors and of their variational approximations
- On the properties of variational approximations of Gibbs posteriors
- \(\alpha\)-variational inference with statistical guarantees
- Some simple formulas for posterior convergence rates
- Convergence rates of posterior distributions.
density estimationempirical Bayesposterior contractionGaussian sequence modelpiecewise constant modelmean-field variational inference
Cites Work
- Simultaneous analysis of Lasso and Dantzig selector
- Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels
- Convergence rates of posterior distributions for non iid observations
- Convergence rates of posterior distributions.
- Concentration of tempered posteriors and of their variational approximations
- Convergence of estimates under dimensionality restrictions
- 10.1162/jmlr.2003.3.4-5.993
- Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies
- On Bayesian supremum norm contraction rates
- Asymptotic behaviour of the empirical Bayes posteriors associated to maximum marginal likelihood estimator
- Rates of convergence of posterior distributions.
- Lower bounds for posterior rates with Gaussian process priors
- Adaptive Bayesian density estimation with location-scale mixtures
- Rates of contraction of posterior distributions based on Gaussian process priors
- The consistency of posterior distributions in nonparametric problems
- On Bayesian consistency
- On Bayes procedures
- Bayesian linear regression with sparse priors
- On adaptive posterior concentration rates
- On rates of convergence for posterior distributions in infinite-dimensional models
- Rényi Divergence and Kullback-Leibler Divergence
- Posterior concentration rates for infinite dimensional exponential families
- Adaptive density estimation for clustering with Gaussian mixtures
- Consistency of variational Bayes inference for estimation and model selection in mixtures
- Rate exact Bayesian adaptation with modified block priors
- \(\alpha\)-variational inference with statistical guarantees
- Frequentist consistency of variational Bayes
Cited In (34)
- Frequentist consistency of variational Bayes
- Variational Bayes for Fast and Accurate Empirical Likelihood Inference
- Tightening bounds for variational inference by revisiting perturbation theory
- Variational Bayes for High-Dimensional Linear Regression With Sparse Priors
- Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model
- Concentration of tempered posteriors and of their variational approximations
- Covariances, robustness, and variational Bayes
- Bayesian Joint Chance Constrained Optimization: Approximations and Statistical Consistency
- Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression
- Rates of convergence of posterior distributions.
- On the properties of variational approximations of Gibbs posteriors
- VAE-KRnet and its applications to variational Bayes
- Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
- Comprehensive study of variational Bayes classification for dense deep neural networks
- Statistical foundation of variational Bayes neural networks
- Heavy-tailed Bayesian nonparametric adaptation
- Adaptive variational Bayes: optimality, computation and applications
- On the approximation accuracy of Gaussian variational inference
- Local convexity of the TAP free energy and AMP convergence for \(\mathbb{Z}_2\)-synchronization
- Black Box Variational Bayesian Model Averaging
- The computational asymptotics of Gaussian variational inference and the Laplace approximation
- Convergence rate of random scan coordinate ascent variational inference under log-concavity
- Bayesian Distributionally Robust Optimization
- Variational Bayes Estimation of Mixing Coefficients
- On variational Bayes estimation and variational information criteria for linear regression models
- Approximating Bayes in the 21st century
- Emerging directions in Bayesian computation
- Spike and slab Bayesian sparse principal component analysis
- Gaussian variational approximations for high-dimensional state space models
- Averaged collapsed variational Bayes inference
- Data-driven priors and their posterior concentration rates
- User-friendly Introduction to PAC-Bayes Bounds
- Non-centered parametric variational Bayes’ approach for hierarchical inverse problems of partial differential equations
- \(\alpha\)-variational inference with statistical guarantees
Uses Software
This page was built for publication: Convergence rates of variational posterior distributions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2215731)
