Gradient explosion free algorithm for training recurrent neural networks

DOI10.12941/JKSIAM.2020.24.331zbMATH Open1475.34031MaRDI QIDQ5013261FDOQ5013261

Authors: Seoyoung Hong, Hyerin Jeon, Byungjoon Lee, Chohong Min 

Publication date: 29 November 2021

Recommendations

• scientific article; zbMATH DE number 7255038
• The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
• Survey of unstable gradients in deep neural network training
• A conjugate gradient learning algorithm for recurrent neural networks
• A novel fractional gradient-based learning algorithm for recurrent neural networks

zbMATH Keywords

dynamical systemsoptimal controlrecurrent neural networksgradient explosion

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Bifurcation theory for ordinary differential equations (34C23) Dynamical systems in numerical analysis (37N30)

Cites Work

• Large-scale machine learning with stochastic gradient descent
• Backpropagation and stochastic gradient descent method

Cited In (4)

• Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm
• A conjugate gradient learning algorithm for recurrent neural networks
• Survey of unstable gradients in deep neural network training
• Sgornn: Combining scalar gates and orthogonal constraints in recurrent networks

This page was built for publication: Gradient explosion free algorithm for training recurrent neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5013261)