Learning finite-dimensional coding schemes with nonlinear reconstruction maps
From MaRDI portal
Optimal transportation (49Q22) Coding and information theory (compaction, compression, models of communication, encoding schemes, etc.) (aspects in computer science) (68P30) Learning and adaptive systems in artificial intelligence (68T05) Artificial neural networks and deep learning (68T07) Signal theory (characterization, reconstruction, filtering, etc.) (94A12) Rate-distortion theory in information and communication theory (94A34)
Abstract: This paper generalizes the Maurer--Pontil framework of finite-dimensional lossy coding schemes to the setting where a high-dimensional random vector is mapped to an element of a compact set of latent representations in a lower-dimensional Euclidean space, and the reconstruction map belongs to a given class of nonlinear maps. Under this setup, which encompasses a broad class of unsupervised representation learning problems, we establish a connection to approximate generative modeling under structural constraints using the tools from the theory of optimal transportation. Next, we consider problem of learning a coding scheme on the basis of a finite collection of training samples and present generalization bounds that hold with high probability. We then illustrate the general theory in the setting where the reconstruction maps are implemented by deep neural nets.
Recommendations
- Dimensionality-dependent generalization bounds for \(k\)-dimensional coding schemes
- Generalization Bounds for K-Dimensional Coding Schemes in Hilbert Spaces
- High-dimensional distribution generation through deep neural networks
- Wasserstein dictionary learning: optimal transport-based unsupervised nonlinear dictionary learning
- A provably convergent scheme for compressive sensing under random generative priors
Cites work
- K-Dimensional Coding Schemes in Hilbert Spaces
- 10.1162/153244302760200713
- A Connection Between Score Matching and Denoising Autoencoders
- Computation of channel capacity and rate-distortion functions
- Concentration inequalities. A nonasymptotic theory of independence
- Constructive quantization: approximation by empirical measures
- Deep learning
- Elements of Information Theory
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 1332320 (Why is no real title available?)
- scientific article; zbMATH DE number 467196 (Why is no real title available?)
- scientific article; zbMATH DE number 1909499 (Why is no real title available?)
- scientific article; zbMATH DE number 3318548 (Why is no real title available?)
- Interactions between compressed sensing random matrices and high dimensional geometry
- Lossy source coding
- Measurable selection theorems for optimization problems
- Neural Network Learning
- On the Performance of Clustering in Hilbert Spaces
- On the rate-distortion function of random vectors and stationary sources with mixed distributions
- Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
- Principal component analysis.
- Quantization and the method of<tex>k</tex>-means
- Sharp asymptotic and finite-sample rates of convergence of empirical measures in Wasserstein distance
- Size-independent sample complexity of neural networks
- Sudakov minoration principle and supremum of some processes
- The minimax distortion redundancy in empirical quantizer design
- The sample complexity of dictionary learning
- The sizes of compact subsets of Hilbert space and continuity of Gaussian processes
- The Speed of Mean Glivenko-Cantelli Convergence
- Upper and lower bounds for stochastic processes. Modern methods and classical problems
Cited in
(5)- Generalization Bounds for K-Dimensional Coding Schemes in Hilbert Spaces
- scientific article; zbMATH DE number 1182506 (Why is no real title available?)
- Redundancy reduction with information-preserving nonlinear maps
- Dimensionality-dependent generalization bounds for \(k\)-dimensional coding schemes
- LSTM-PINN: an hybrid method for prediction of steady-state electrohydrodynamic flow
This page was built for publication: Learning finite-dimensional coding schemes with nonlinear reconstruction maps
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5025792)
