data compressiondeep learninginformation theoryrate-distortion theorysource codingspeech/audio/image/video compressionvariational inference
Coding and information theory (compaction, compression, models of communication, encoding schemes, etc.) (aspects in computer science) (68P30) Artificial neural networks and deep learning (68T07) Machine vision and scene understanding (68T45) Source coding (94A29) Rate-distortion theory in information and communication theory (94A34)
Abstract: Neural compression is the application of neural networks and other machine learning methods to data compression. Recent advances in statistical machine learning have opened up new possibilities for data compression, allowing compression algorithms to be learned end-to-end from data using powerful generative models such as normalizing flows, variational autoencoders, diffusion probabilistic models, and generative adversarial networks. The present article aims to introduce this field of research to a broader machine learning audience by reviewing the necessary background in information theory (e.g., entropy coding, rate-distortion theory) and computer vision (e.g., image quality assessment, perceptual metrics), and providing a curated guide through the essential ideas and methods in the literature thus far.
Recommendations
- scientific article; zbMATH DE number 2221026
- Multilayer perceptrons and data compression
- Lossless compression of deep neural networks
- Information-theoretic self-compression of multi-layered neural networks
- Survey of deep neural network model compression
- Estimates on compressed neural networks regression
Cites work
- A Feature-Enriched Completely Blind Image Quality Evaluator
- A Mathematical Theory of Communication
- A Method for the Construction of Minimum-Redundancy Codes
- A universal algorithm for sequential data compression
- An algorithm for computing the capacity of arbitrary discrete memoryless channels
- Arithmetic Coding
- Auto-association by multilayer perceptrons and singular value decomposition
- Computation of channel capacity and rate-distortion functions
- Deep learning
- Discrete Cosine Transform
- Efficient scalar quantization of exponential and Laplacian random variables
- Elements of Information Theory
- Entanglement-assisted capacity of a quantum channel and the reverse Shannon theorem
- scientific article; zbMATH DE number 2061729 (Why is no real title available?)
- Lattice coding for signals and networks. A structured coding approach to quantization, modulation, and multiuser information theory. With contributions by Bobak Nazer and Yuval Kochman
- Least squares quantization in PCM
- Lossy source coding
- On surrogate loss functions and \(f\)-divergences
- On universal quantization
- On universal quantization by randomized uniform/lattice quantizers
- Pattern recognition and machine learning.
- Quantization
- Simple statistical gradient-following algorithms for connectionist reinforcement learning
- The Likelihood Encoder for Lossy Compression
- Transform coding with backward adaptive updates
Cited in
(7)- Information-theoretic self-compression of multi-layered neural networks
- Multimodal foundation models: from specialists to general-purpose assistants
- Neural networks generalize on low complexity data
- Channel simulation: theory and applications to lossy compression and differential privacy
- Tackling challenge of masked and unmasked face for facial recognition through dimensionality reduction and deep learning
- Data with logical and statistical constraints
- Estimates on compressed neural networks regression
This page was built for publication: An Introduction to Neural Data Compression
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6180224)
