Deep nonparametric estimation of intrinsic data structures by chart autoencoders: generalization error and robustness
From MaRDI portal
(Redirected from Publication:6144890)
Abstract: Autoencoders have demonstrated remarkable success in learning low-dimensional latent features of high-dimensional data across various applications. Assuming that data are sampled near a low-dimensional manifold, we employ chart autoencoders, which encode data into low-dimensional latent features on a collection of charts, preserving the topology and geometry of the data manifold. Our paper establishes statistical guarantees on the generalization error of chart autoencoders, and we demonstrate their denoising capabilities by considering noisy training samples, along with their noise-free counterparts, on a -dimensional manifold. By training autoencoders, we show that chart autoencoders can effectively denoise the input data with normal noise. We prove that, under proper network architectures, chart autoencoders achieve a squared generalization error in the order of , which depends on the intrinsic dimension of the manifold and only weakly depends on the ambient dimension and noise level. We further extend our theory on data with noise containing both normal and tangential components, where chart autoencoders still exhibit a denoising effect for the normal component. As a special case, our theory also applies to classical autoencoders, as long as the data manifold has a global parametrization. Our results provide a solid theoretical foundation for the effectiveness of autoencoders, which is further validated through several numerical experiments.
Recommendations
- Generalization error guaranteed auto-encoder-based nonlinear model reduction for operator learning
- Convergent autoencoder approximation of low bending and low distortion manifold embeddings
- Nonparametric regression on low-dimensional manifolds using deep ReLU networks: function approximation and statistical recovery
- Understanding autoencoders with information theoretic concepts
- Adaptive approximation and generalization of deep neural network with intrinsic dimensionality
Cites work
- 50 years sets with positive reach -- a survey
- A deep network construction that adapts to intrinsic dimensionality beyond the domain
- A distribution-free theory of nonparametric regression
- Accurate uncertainty quantification using inaccurate computational models
- Adaptive approximation and generalization of deep neural network with intrinsic dimensionality
- Adaptive geometric multiscale approximations for intrinsically low-dimensional data
- Adaptive two-layer ReLU neural network. I: Best least-squares approximation
- Auto-association by multilayer perceptrons and singular value decomposition
- Curvature Measures
- Error bounds for approximations with deep ReLU networks
- Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
- Finding the homology of submanifolds with high confidence from random samples
- Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps
- scientific article; zbMATH DE number 3479703 (Why is no real title available?)
- scientific article; zbMATH DE number 7625193 (Why is no real title available?)
- Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
- Learning deep architectures for AI
- Manifold estimation and singular deconvolution under Hausdorff loss
- Manifold reconstruction using tangential Delaunay complexes
- Minimax manifold estimation
- Multi-fidelity Gaussian process regression for prediction of random fields
- Multi-scale geometric methods for data sets. II: Geometric multi-resolution analysis
- Multilayer feedforward networks are universal approximators
- Nonasymptotic rates for manifold, tangent space and curvature estimation
- Provable approximation properties for deep neural networks
- Side effects of learning from low-dimensional data embedded in a Euclidean space
- Testing the manifold hypothesis
- The reach, metric distortion, geodesic convexity and the variation of tangent spaces
- Visualizing data using t-SNE
- Weak convergence and empirical processes. With applications to statistics
Cited in
(3)
This page was built for publication: Deep nonparametric estimation of intrinsic data structures by chart autoencoders: generalization error and robustness
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6144890)
