The limited-memory recursive variational Gaussian approximation (L-RVGA)
From MaRDI portal
Abstract: We consider the problem of computing a Gaussian approximation to the posterior distribution of a parameter given a large number N of observations and a Gaussian prior, when the dimension of the parameter d is also large. To address this problem we build on a recently introduced recursive algorithm for variational Gaussian approximation of the posterior, called recursive variational Gaussian approximation (RVGA), which is a single pass algorithm, free of parameter tuning. In this paper, we consider the case where the parameter dimension d is high, and we propose a novel version of RVGA that scales linearly in the dimension d (as well as in the number of observations N), and which only requires linear storage capacity in d. This is afforded by the use of a novel recursive expectation maximization (EM) algorithm applied for factor analysis introduced herein, to approximate at each step the covariance matrix of the Gaussian distribution conveying the uncertainty in the parameter. The approach is successfully illustrated on the problems of high dimensional least-squares and logistic regression, and generalized to a large class of nonlinear models.
Recommendations
- The recursive variational Gaussian approximation (R-VGA)
- The Variational Gaussian Approximation Revisited
- Gaussian variational approximation with sparse precision matrices
- Asymptotic normality and valid inference for Gaussian variational approximation
- Variational Gaussian approximation for Poisson data
Cites work
- A Stochastic Approximation Method
- Adaptive subgradient methods for online learning and stochastic optimization
- An introduction to MCMC for machine learning
- Bayesian reasoning and machine learning.
- EM algorithms for ML factor analysis
- Gaussian Kullback-Leibler approximate inference
- Gaussian Variational Approximation With a Factor Covariance Structure
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 1222283 (Why is no real title available?)
- scientific article; zbMATH DE number 1222290 (Why is no real title available?)
- scientific article; zbMATH DE number 1304260 (Why is no real title available?)
- scientific article; zbMATH DE number 1569116 (Why is no real title available?)
- scientific article; zbMATH DE number 7306852 (Why is no real title available?)
- Information Theory and Statistics: A Tutorial
- On-line expectation-maximization algorithm for latent data models
- Online natural gradient as a Kalman filter
- Probabilistic Principal Component Analysis
- Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems
- Stochastic processes and filtering theory
- The recursive variational Gaussian approximation (R-VGA)
- The Variational Gaussian Approximation Revisited
- Variational inference for generalized linear mixed models using partially noncentered parametrizations
Cited in
(5)- Gaussian variational approximation with sparse precision matrices
- The recursive variational Gaussian approximation (R-VGA)
- scientific article; zbMATH DE number 1640103 (Why is no real title available?)
- R-VGAL: a sequential variational Bayes algorithm for generalised linear mixed models
- Eigengap sparsity for covariance parsimony
This page was built for publication: The limited-memory recursive variational Gaussian approximation (L-RVGA)
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6172928)
