Survey of unstable gradients in deep neural network training

DOI10.13328/J.CNKI.JOS.005561zbMATH Open1424.68116MaRDI QIDQ4624686FDOQ4624686

Authors: J. T. Chen, Yang Xiang 

Publication date: 22 February 2019

Recommendations

• Optimization for deep learning: an overview
• Non-convergence of stochastic gradient descent in the training of deep neural networks
• Gradient explosion free algorithm for training recurrent neural networks
• Why does large batch training result in poor generalization? A comprehensive explanation and a better strategy from the viewpoint of stochastic optimization
• Research progress on batch normalization of deep learning and its related algorithms

zbMATH Keywords

deep neural networkvanishing gradientexploding gradientunstable gradient

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Cited In (4)

• Gradient explosion free algorithm for training recurrent neural networks
• Research on the effect of batch normalization on VGG-like neural networks
• On the S-instability and degeneracy of discrete deep learning models
• Research progress on batch normalization of deep learning and its related algorithms

This page was built for publication: Survey of unstable gradients in deep neural network training

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4624686)