Geometrical Insights for Implicit Generative Modeling

DOI10.1007/978-3-319-99492-5_11MaRDI QIDQ6162298zbMATH OpenOpenAlexFDO

Authors Léon Bottou, David Lopez-Paz, Maxime Oquab

Publication date 28 June 2023

Published in Braverman Readings in Machine Learning. Key Ideas from Inception to Current State (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1712.07822

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Learning algorithms for implicit generative models can optimize a variety of criteria that measure how the data distribution differs from the implicit model distribution, including the Wasserstein distance, the Energy distance, and the Maximum Mean Discrepancy criterion. A careful look at the geometries induced by these distances on the space of probability measures reveals interesting differences. In particular, we can establish surprising approximate global convergence guarantees for the $1$-Wasserstein distance,even when the parametric generator has a nonconvex parametrization.

Recommendations

• A geometric view of optimal transportation and generative model
• On the capacity of deep generative networks for approximating distributions
• Optimal transport for generative models
• Multi-projection of unequal dimension optimal transport theory for generative adversary networks
• Some theoretical properties of GANs

Cites work

• scientific article; zbMATH DE number 3103824 (Why is no real title available?)
• A characterization of distributions by mean values of statistics and certain probabilistic metrics
• A course in metric geometry
• A kernel two-sample test
• Approximation Theorems of Mathematical Statistics
• Complexity of random smooth functions on the high-dimensional sphere
• Constructive quantization: approximation by empirical measures
• Convergence rate of linear two-time-scale stochastic approximation.
• Energy statistics: a class of statistics based on distances
• Envelope Theorems for Arbitrary Choice Sets
• Equivalence of distance-based and RKHS-based statistics in hypothesis testing
• Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization
• Gluing lemmas and Skorohod representations
• Integral Probability Metrics and Their Generating Classes of Functions
• On the Asymptotic Distribution of Differentiable Statistical Functions
• On the empirical estimation of integral probability metrics
• On the optimal estimation of probability measures in weak and strong topologies
• On the rate of convergence in Wasserstein distance of the empirical measure
• Optimal Transport
• Stochastic approximation with two time scales
• The Distribution of Distance in a Hypersphere
• The methods of distances in the theory of probability and statistics.
• Theoretical foundations of the potential function method in pattern recognition learning
• Theory of Reproducing Kernels
• Universality, Characteristic Kernels and RKHS Embedding of Measures

Cited in

(6)

• Processing simple geometric attributes with autoencoders
• Radon-Sobolev variational auto-encoders
• scientific article; zbMATH DE number 5066628 (Why is no real title available?)
• Spurious valleys in one-hidden-layer neural network optimization landscapes
• Characterization of Gromov-type geodesics
• The ultrametric Gromov-Wasserstein distance

This page was built for publication: Geometrical Insights for Implicit Generative Modeling

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6162298)