Emergence of invariance and disentanglement in deep representations

MaRDI QIDQ4558200zbMATH OpenFDO

Authors Alessandro Achille, Stefano Soatto

Publication date 21 November 2018

Full work available at URL https://arxiv.org/abs/1706.01350, http://jmlr.csail.mit.edu/papers/v19/17-646.html

zbMATH Keywords

flat minimageneralizationindependenceinformation bottleneckinvariancePAC-Bayesrepresentation learning

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Bayesian inference (62F15) Artificial neural networks and deep learning (68T07)

Recommendations

• Entropy and mutual information in models of deep neural networks*
• On the information bottleneck theory of deep learning
• Neural learning methods yielding functional invariance
• scientific article; zbMATH DE number 1843021
• Group invariance, stability to deformations, and complexity of deep convolutional representations

Cites work

• Actionable Information in Vision
• Emergence of invariance and disentanglement in deep representations
• Flat Minima
• scientific article; zbMATH DE number 3895680 (Why is no real title available?)
• scientific article; zbMATH DE number 107482 (Why is no real title available?)
• scientific article; zbMATH DE number 523032 (Why is no real title available?)
• Learning deep architectures for AI
• On invariance and selectivity in representation learning
• Provable learning of noisy-OR networks
• Some large-scale matrix computation problems
• The elements of statistical learning. Data mining, inference, and prediction

Cited in

(23)

• DEFT: distilling entangled factors by preventing information diffusion
• Neural collapse with unconstrained features
• Few-shot learning with adaptively initialized task optimizer: a practical meta-learning approach
• Understanding autoencoders with information theoretic concepts
• On the benefits of representation regularization in invariance based domain generalization
• Exploring the common principal subspace of deep features in neural networks
• Unsupervised learning of invariant representations
• Emergence of invariance and disentanglement in deep representations
• On invariance and selectivity in representation learning
• scientific article; zbMATH DE number 1843021 (Why is no real title available?)
• Artificial neural variability for deep learning: on overfitting, noise memorization, and catastrophic forgetting
• Categorical perception: a groundwork for deep learning
• scientific article; zbMATH DE number 7626742 (Why is no real title available?)
• Habituation as a neural algorithm for online odor discrimination
• Invariance, Encodings, and Generalization: Learning Identity Effects With Neural Networks
• Prediction, estimation, and attribution
• Understanding Deep Learning with Statistical Relevance
• Neural learning methods yielding functional invariance
• Prediction, Estimation, and Attribution
• Unsupervised learning of disentangled representations in deep restricted kernel machines with orthogonality constraints
• The role of mutual information in variational classifiers
• Redundant representations help generalization in wide neural networks
• Generalization bounds: perspectives from information theory and PAC-Bayes

Describes a project that uses

Uses Software

• CIFAR
• Entropy-SGD

This page was built for publication: Emergence of invariance and disentanglement in deep representations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4558200)