Asymptotic Consistency of \alpha-R\'enyi-Approximate Posteriors
From MaRDI portal
Publication:6313645
zbMATH Open1525.62016arXiv1902.01902MaRDI QIDQ6313645FDOQ6313645
Authors: Prateek Jaiswal, Harsha Honnappa 
Publication date: 5 February 2019
Abstract: We study the asymptotic consistency properties of -R'enyi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the -R'enyi divergence from the true posterior. Unique to our work is that we consider settings with , resulting in approximations that upperbound the log-likelihood, and consequently have wider spread than traditional variational approaches that minimize the Kullback-Liebler (KL) divergence from the posterior. Our primary result identifies sufficient conditions under which consistency holds, centering around the existence of a 'good' sequence of distributions in the approximating family that possesses, among other properties, the right rate of convergence to a limit distribution. We further characterize the good sequence by demonstrating that a sequence of distributions that converges too quickly cannot be a good sequence. We also extend our analysis to the setting where equals one, corresponding to the minimizer of the reverse KL divergence, and to models with local latent variables. We also illustrate the existence of good sequence with a number of examples. Our results complement a growing body of work focused on the frequentist properties of variational Bayesian methods.
Computational methods for problems pertaining to statistics (62-08) Asymptotic properties of parametric estimators (62F12) Bayesian inference (62F15) Learning and adaptive systems in artificial intelligence (68T05)
This page was built for publication: Asymptotic Consistency of $\alpha-$R\'enyi-Approximate Posteriors
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6313645)
