Accelerated information gradient flow
From MaRDI portal
Abstract: We present a framework for Nesterov's accelerated gradient flows in probability space to design efficient mean-field Markov chain Monte Carlo (MCMC) algorithms for Bayesian inverse problems. Here four examples of information metrics are considered, including Fisher-Rao metric, Wasserstein-2 metric, Kalman-Wasserstein metric and Stein metric. For both Fisher-Rao and Wasserstein-2 metrics, we prove convergence properties of accelerated gradient flows. In implementations, we propose a sampling-efficient discrete-time algorithm for Wasserstein-2, Kalman-Wasserstein and Stein accelerated gradient flows with a restart technique. We also formulate a kernel bandwidth selection method, which learns the gradient of logarithm of density from Brownian-motion samples. Numerical experiments, including Bayesian logistic regression and Bayesian neural network, show the strength of the proposed methods compared with state-of-the-art algorithms.
Recommendations
- Is there an analog of Nesterov acceleration for gradient-based MCMC?
- Natural gradient via optimal transport
- The Bayesian update: variational formulations and gradient flows
- A note on parametric Bayesian inference via gradient flows
- Projected Wasserstein Gradient Descent for High-Dimensional Bayesian Inference
Cites work
- A blob method for diffusion
- A Mean-Field Optimal Control Formulation for Global Optimization
- A variational perspective on accelerated methods in optimization
- Adaptive restart for accelerated gradient schemes
- Convergence to equilibrium in Wasserstein distance for damped Euler equations with interaction forces
- Functional and shape data analysis
- Geometry of matrix decompositions seen through optimal transport and information geometry
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 44578 (Why is no real title available?)
- scientific article; zbMATH DE number 1909499 (Why is no real title available?)
- scientific article; zbMATH DE number 877623 (Why is no real title available?)
- Improvement on some known nonparametric uniformly consistent estimators of derivatives of a density
- Information geometry and its applications
- Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
- Inverse problems: a Bayesian perspective
- Is there an analog of Nesterov acceleration for gradient-based MCMC?
- On Wasserstein geometry of the space of Gaussian measures
- The Density Manifold and Configuration Space Quantization
- THE GEOMETRY OF DISSIPATIVE EVOLUTION EQUATIONS: THE POROUS MEDIUM EQUATION
- Wasserstein Hamiltonian flows
- Wasserstein Riemannian geometry of Gaussian densities
Cited in
(25)- Natural gradient via optimal transport
- Principal whitened gradient for information geometry
- Is there an analog of Nesterov acceleration for gradient-based MCMC?
- Sparse optimization on measures with over-parameterized gradient descent
- A note on parametric Bayesian inference via gradient flows
- The Bayesian update: variational formulations and gradient flows
- Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
- Global Optimization via Schrödinger–Föllmer Diffusion
- High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems
- Scaling Up Bayesian Uncertainty Quantification for Inverse Problems Using Deep Neural Networks
- A Time-Fractional Mean-Field Control Modeling Subdiffusive Advective Transport
- Lagrangian and Hamiltonian dynamics for probabilities on the statistical bundle
- Hölder stability and uniqueness for the mean field games system via Carleman estimates
- Noise-free sampling algorithms via regularized Wasserstein proximals
- Accelerated Stein variational gradient flow
- Transporting higher-order quadrature rules -- quasi-Monte Carlo points and sparse grids for mixture distributions
- A gradient-based optimization method using the Koopman operator
- Score-based neural ordinary differential equations for computing mean field control problems
- Gradient-adjusted underdamped Langevin dynamics for sampling
- Nesterov acceleration for ensemble Kalman inversion and variants
- Tensor train based sampling algorithms for approximating regularized Wasserstein proximal operators
- Deep learning for PDE-based inverse problems. Abstracts from the workshop held October 27 -- November 1, 2024
- Accelerating optimization over the space of probability measures
- A particle algorithm for mean-field variational inference
- Flows on measure spaces and applications in machine learning. Abstracts from the workshop held March 22--27, 2026
This page was built for publication: Accelerated information gradient flow
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2053344)
