Optimal transport natural gradient for statistical manifolds with continuous sample space
From MaRDI portal
Publication:2006192
DOI10.1007/S41884-020-00028-0zbMATH Open1450.62009arXiv1805.08380OpenAlexW3025478249MaRDI QIDQ2006192FDOQ2006192
Authors: Wuchen Li, Yifan Chen 
Publication date: 8 October 2020
Published in: Information Geometry (Search for Journal in Brave)
Abstract: We study the Wasserstein natural gradient in parametric statistical models with continuous sample spaces. Our approach is to pull back the -Wasserstein metric tensor in the probability density space to a parameter space, equipping the latter with a positive definite metric tensor, under which it becomes a Riemannian manifold, named the Wasserstein statistical manifold. In general, it is not a totally geodesic sub-manifold of the density space, and therefore its geodesics will differ from the Wasserstein geodesics, except for the well-known Gaussian distribution case, a fact which can also be validated under our framework. We use the sub-manifold geometry to derive a gradient flow and natural gradient descent method in the parameter space. When parametrized densities lie in , the induced metric tensor establishes an explicit formula. In optimization problems, we observe that the natural gradient descent outperforms the standard gradient descent when the Wasserstein distance is the objective function. In such a case, we prove that the resulting algorithm behaves similarly to the Newton method in the asymptotic regime. The proof calculates the exact Hessian formula for the Wasserstein distance, which further motivates another preconditioner for the optimization process. To the end, we present examples to illustrate the effectiveness of the natural gradient in several parametric statistical models, including the Gaussian measure, Gaussian mixture, Gamma distribution, and Laplace distribution.
Full work available at URL: https://arxiv.org/abs/1805.08380
Recommendations
Statistics on manifolds (62R30) Measures of information, entropy (94A17) Differential geometric aspects of statistical manifolds and information geometry (53B12) Information geometry (statistical aspects) (62B11)
Cites Work
- Title not available (Why is that?)
- Computational optimal transport. With applications to data sciences
- A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem
- Wasserstein geometry of Gaussian measures
- Ricci curvature for metric-measure spaces via optimal transport
- THE GEOMETRY OF DISSIPATIVE EVOLUTION EQUATIONS: THE POROUS MEDIUM EQUATION
- Title not available (Why is that?)
- Large-scale dynamics of mean-field games driven by local Nash equilibria
- Constrained steepest descent in the 2-Wasserstein metric
- Natural gradient flow in the mixture geometry of a discrete exponential family
- Information-geometric optimization algorithms: a unifying picture via invariance principles
- Some geometric calculations on Wasserstein space
- Stability of a 4th-order curvature condition arising in optimal transport theory
- Optimal transport for seismic full waveform inversion
- The Density Manifold and Configuration Space Quantization
- Optimal algorithms for online scheduling with bounded rearrangement at the end
- Application of the Wasserstein metric to seismic signals
- Wasserstein Riemannian geometry of Gaussian densities
- Information geometry and its applications
- The quadratic Wasserstein metric for earthquake location
- Natural gradient via optimal transport
- Information geometry
- An optimal transport approach for seismic tomography: application to 3D full waveform inversion
- Information geometry connecting Wasserstein distance and Kullback-Leibler divergence via the entropy-relaxed transportation problem
- Logarithmic divergences from optimal transport and Rényi geometry
- Geometry of matrix decompositions seen through optimal transport and information geometry
- Online natural gradient as a Kalman filter
- Population games and discrete optimal transport
- Robust estimation of natural gradient in optimization by regularized linear regression
Cited In (12)
- Quantum statistical learning via quantum Wasserstein natural gradient
- Natural gradient for combined loss using wavelets
- Wasserstein information matrix
- Affine natural proximal learning
- Mean-field and kinetic descriptions of neural differential equations
- High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems
- Information geometry of Wasserstein statistics on shapes and affine deformations
- When optimal transport meets information geometry
- Efficient Natural Gradient Descent Methods for Large-Scale PDE-Based Optimization Problems
- Title not available (Why is that?)
- Natural gradient via optimal transport
- Information geometry connecting Wasserstein distance and Kullback-Leibler divergence via the entropy-relaxed transportation problem
Uses Software
This page was built for publication: Optimal transport natural gradient for statistical manifolds with continuous sample space
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2006192)
