Chaining meets chain rule: multilevel entropic regularization and training of neural networks

MaRDI QIDQ4969253zbMATH OpenFDO

Authors Amir R. Asadi, Emmanuel Abbe

Publication date 5 October 2020

Full work available at URL https://arxiv.org/abs/1906.11148, https://jmlr.csail.mit.edu/papers/v21/19-794.html

zbMATH Keywords

neural networkschaining mutual informationmultilevel relative entropymultiscale generalization boundmultiscale Gibbs distribution

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• Entropy and mutual information in models of deep neural networks*
• Riemannian metrics for neural networks. I: Feedforward networks
• Entropy-SGD: biasing gradient descent into wide valleys
• Mean Field Analysis of Deep Neural Networks
• Globally Convergent Multilevel Training of Deep Residual Networks

Cites work

• scientific article; zbMATH DE number 5544465 (Why is no real title available?)
• scientific article; zbMATH DE number 3595981 (Why is no real title available?)
• scientific article; zbMATH DE number 6781368 (Why is no real title available?)
• Fifty years of Shannon theory
• From -entropy to KL-entropy: analysis of minimum information complexity density estima\-tion
• High-dimensional probability. An introduction with applications in data science
• Information-theoretic upper and lower bounds for statistical estimation
• Learners that use little information
• On prediction of individual sequences
• On the properties of variational approximations of Gibbs posteriors
• Probability and stochastics.
• Rényi Divergence and Kullback-Leibler Divergence
• Sparse estimation by exponential weighting
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
• Upper and lower bounds for stochastic processes. Modern methods and classical problems

Cited in

(2)

• Entropic regularization of neural networks: self-similar approximations
• Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Nets

This page was built for publication: Chaining meets chain rule: multilevel entropic regularization and training of neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4969253)