Survey of unstable gradients in deep neural network training

DOI10.13328/J.CNKI.JOS.005561MaRDI QIDQ4624686zbMATH OpenFDO

Authors J. T. Chen, Yang Xiang

Publication date 22 February 2019

zbMATH Keywords

deep neural networkvanishing gradientexploding gradientunstable gradient

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• Optimization for deep learning: an overview
• Non-convergence of stochastic gradient descent in the training of deep neural networks
• Gradient explosion free algorithm for training recurrent neural networks
• Why does large batch training result in poor generalization? A comprehensive explanation and a better strategy from the viewpoint of stochastic optimization
• Research progress on batch normalization of deep learning and its related algorithms

Cited in

(4)

• Gradient explosion free algorithm for training recurrent neural networks
• Research on the effect of batch normalization on VGG-like neural networks
• On the S-instability and degeneracy of discrete deep learning models
• Research progress on batch normalization of deep learning and its related algorithms

This page was built for publication: Survey of unstable gradients in deep neural network training

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4624686)