Gradient explosion free algorithm for training recurrent neural networks

DOI10.12941/JKSIAM.2020.24.331MaRDI QIDQ5013261zbMATH OpenFDO

Authors Seoyoung Hong, Hyerin Jeon, Byungjoon Lee, Chohong Min

Publication date 29 November 2021

Full work available at URL https://doi.org/10.12941/JKSIAM.2020.24.331

zbMATH Keywords

dynamical systemsgradient explosionoptimal controlrecurrent neural networks

Mathematics Subject Classification ID

Bifurcation theory for ordinary differential equations (34C23) Dynamical systems in numerical analysis (37N30) Artificial neural networks and deep learning (68T07)

Recommendations

• scientific article; zbMATH DE number 7255038
• The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
• Survey of unstable gradients in deep neural network training
• A conjugate gradient learning algorithm for recurrent neural networks
• A novel fractional gradient-based learning algorithm for recurrent neural networks

Cites work

• Backpropagation and stochastic gradient descent method
• Large-scale machine learning with stochastic gradient descent

Cited in

(4)

• A conjugate gradient learning algorithm for recurrent neural networks
• Survey of unstable gradients in deep neural network training
• Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm
• Sgornn: Combining scalar gates and orthogonal constraints in recurrent networks

This page was built for publication: Gradient explosion free algorithm for training recurrent neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5013261)