Natural gradient via optimal transport
From MaRDI portal
Publication:1713655
DOI10.1007/S41884-018-0015-3zbMATH Open1409.62022arXiv1803.07033OpenAlexW2963645788WikidataQ126033651 ScholiaQ126033651MaRDI QIDQ1713655FDOQ1713655
Authors: Wuchen Li, Guido Montúfar 
Publication date: 28 January 2019
Published in: Information Geometry (Search for Journal in Brave)
Abstract: We study a natural Wasserstein gradient flow on manifolds of probability distributions with discrete sample spaces. We derive the Riemannian structure for the probability simplex from the dynamical formulation of the Wasserstein distance on a weighted graph. We pull back the geometric structure to the parameter space of any given probability model, which allows us to define a natural gradient flow there. In contrast to the natural Fisher-Rao gradient, the natural Wasserstein gradient incorporates a ground metric on sample space. We illustrate the analysis of elementary exponential family examples and demonstrate an application of the Wasserstein natural gradient to maximum likelihood estimation.
Full work available at URL: https://arxiv.org/abs/1803.07033
Recommendations
machine learningoptimal transportdisplacement convexityinformation geometrymanifolds of probability distributionsWasserstein statistical manifold
Probability distributions: general theory (60E05) Statistical aspects of information-theoretic topics (62B10) Learning and adaptive systems in artificial intelligence (68T05) Applications of global differential geometry to the sciences (53C80)
Cites Work
- Title not available (Why is that?)
- Optimal Transport
- A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem
- Wasserstein geometry of Gaussian measures
- THE GEOMETRY OF DISSIPATIVE EVOLUTION EQUATIONS: THE POROUS MEDIUM EQUATION
- Title not available (Why is that?)
- Fokker-Planck equations for a free energy functional or Markov process on a graph
- A gradient structure for reaction–diffusion systems and for energy-drift-diffusion systems
- Gradient flows of the entropy for finite Markov chains
- Constrained steepest descent in the 2-Wasserstein metric
- Information geometry of Wasserstein divergence
- Title not available (Why is that?)
- Natural gradient flow in the mixture geometry of a discrete exponential family
- On the Fisher metric of conditional probability polytopes
- Axiomatic Geometry of Conditional Models
- Some geometric calculations on Wasserstein space
- The Density Manifold and Configuration Space Quantization
- An Extended Cencov Characterization of the Information Metric
- Information geometry and its applications
- Computations of optimal transport distance with Fisher information regularization
- Information geometry
- Entropy dissipation of Fokker-Planck equations on graphs
- Information geometry connecting Wasserstein distance and Kullback-Leibler divergence via the entropy-relaxed transportation problem
- Ricci curvature for parametric statistics via optimal transport
- Geometry of matrix decompositions seen through optimal transport and information geometry
- Geodesics of minimal length in the set of probability measures on graphs
- Towards the geometry of estimation of distribution algorithms based on the exponential family
Cited In (33)
- Analysis of asymptotic escape of strict saddle sets in manifold optimization
- Affine statistical bundle modeled on a Gaussian Orlicz-Sobolev space
- Geometry and convergence of natural policy gradient methods
- Ricci curvature for parametric statistics via optimal transport
- Optimal transport natural gradient for statistical manifolds with continuous sample space
- Wasserstein proximal of GANs
- Quantum statistical learning via quantum Wasserstein natural gradient
- Neural parametric Fokker-Planck equation
- Wasserstein gradients for the temporal evolution of probability distributions
- Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
- Natural gradient for combined loss using wavelets
- Weyl geometric approach to the gradient-flow equations in information geometry
- Wasserstein information matrix
- Mirror descent algorithms for minimizing interacting free energy
- The information geometry of mirror descent
- Affine natural proximal learning
- Pseudo-Riemannian geometry encodes information geometry in optimal transport
- Transport information geometry: Riemannian calculus on probability simplex
- Hessian transport gradient flows
- Lagrangian and Hamiltonian dynamics for probabilities on the statistical bundle
- Information geometry of physics-informed statistical manifolds and its use in data assimilation
- Information geometry of smooth densities on the Gaussian space: Poincaré inequalities
- Online natural gradient as a Kalman filter
- Accelerated information gradient flow
- High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems
- Transport information Bregman divergences
- Fisher information regularization schemes for Wasserstein gradient flows
- When optimal transport meets information geometry
- Gradient flow of the stochastic relaxation on a generic exponential family
- Hessian metric via transport information geometry
- Sparse optimization on measures with over-parameterized gradient descent
- Invariance properties of the natural gradient in overparametrised systems
- Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems
Uses Software
This page was built for publication: Natural gradient via optimal transport
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1713655)
