Gradient explosion free algorithm for training recurrent neural networks

DOI10.12941/JKSIAM.2020.24.331MaRDI QIDQ5013261zbMATH OpenFDO

Authors Seoyoung Hong, Hyerin Jeon, Byungjoon Lee, Chohong Min

Publication date 29 November 2021

zbMATH Keywords

dynamical systemsoptimal controlrecurrent neural networksgradient explosion

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Bifurcation theory for ordinary differential equations (34C23) Dynamical systems in numerical analysis (37N30)

Recommendations

• scientific article; zbMATH DE number 7255038
• The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
• Survey of unstable gradients in deep neural network training
• A conjugate gradient learning algorithm for recurrent neural networks
• A novel fractional gradient-based learning algorithm for recurrent neural networks

Cites work

• Backpropagation and stochastic gradient descent method
• Large-scale machine learning with stochastic gradient descent

Cited in

(4)

• Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm
• Survey of unstable gradients in deep neural network training
• Sgornn: Combining scalar gates and orthogonal constraints in recurrent networks
• A conjugate gradient learning algorithm for recurrent neural networks

This page was built for publication: Gradient explosion free algorithm for training recurrent neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5013261)