Deep generative models: complexity, dimensionality, and approximation
approximation theorygenerative adversarial networksmachine learningmanifold hypothesisspace-filling curve
Approximation by other special function classes (41A30) 2-dimensional topology (including mapping class groups of surfaces, Teichmüller theory, curve complexes, etc.) (57K20) Learning and adaptive systems in artificial intelligence (68T05) Artificial neural networks and deep learning (68T07) Computational aspects of data analysis and big data (68T09)
This interesting paper studies deep generative models, namely their complexity, dimensionality and approximation. Indeed, generative models have in recent years become a central topic in machine learning. Examples include adversarial networks and variational encoders with applications in a vast array of areas including image registration, anomaly detection, data augmentation and many others. The theoretical advancements in deep generative models, parallelling their practical applications, have been substantial. Although practical applications of these models are well studied and observed, there are significant gaps in theoretical frameworks. One such gap involves the well known manifold hypothesis which posits that real-world high-dimensional data often lives on lower-dimensional manifolds. This concept is critical because it suggests a fundamental reason why models which typically function in low-dimensional spaces, are able to capture complex data distributions effectively. Under the manifold hypothesis, it was believed up till now, that to approximate a distribution on a \(d\)-dimensional Riemannian manifold, the latent dimension needs to be at least \(d\) or \(d + 1\). The authors show, in particular, that this requirement on the latent dimension is not necessary by demonstrating that generative networks can approximate distributions on \(d\)-dimensional Riemannian manifolds from inputs of any dimension for example lower than \(d\) using methods of space filling curves. This idea leads to a super-exponential complexity bound of the deep neural networks through expanded neurons.\N\NThe paper is well written with a good set of references.
- A localization property of viscosity solutions to the Monge-Ampère equation and their strict convexity
- Boundary regularity of maps with convex potentials
- Boundary regularity of maps with convex potentials. II
- scientific article; zbMATH DE number 3960432 (Why is no real title available?)
- scientific article; zbMATH DE number 52737 (Why is no real title available?)
- scientific article; zbMATH DE number 3533448 (Why is no real title available?)
- Interior \(W^{2,p}\) estimates for solutions of the Monge-Ampère equation
- Multilayer feedforward networks are universal approximators
- On the capacity of deep generative networks for approximating distributions
- On the Volume of Tubes
- Optimal Transport
- Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance
- Some regularity properties of solutions of Monge Ampère equation
- Statistical guarantees for generative models without domination
- Testing the manifold hypothesis
- The Regularity of Mappings with a Convex Potential
- Tubes
- Visual Feature Extraction by a Multilayered Network of Analog Threshold Elements
This page was built for publication: Deep generative models: complexity, dimensionality, and approximation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6887414)
