A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures

DOI10.1162/neco_a_01199zbMath1494.68236OpenAlexW2944851425WikidataQ92194256 ScholiaQ92194256MaRDI QIDQ5214344

Yong Yu, Chang-Hua Hu, Jian-Xun Zhang, Xiao-Sheng Si

Publication date: 7 February 2020

Published in: Neural Computation (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1162/neco_a_01199

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20)

Related Items

Codimension-2 parameter space structure of continuous-time recurrent neural networks ⋮ Machine Learning Surrogate Modeling for Meshless Methods: Leveraging Universal Approximation ⋮ A two-stage deep learning architecture for model reduction of parametric time-dependent problems ⋮ Design of continuous-time recurrent neural networks with piecewise-linear activation function for generation of prescribed sequences of bipolar vectors ⋮ Reduced-order autodifferentiable ensemble Kalman filters ⋮ Deep learning-based parameter estimation of stochastic differential equations driven by fractional Brownian motions with measurement noise ⋮ Thermodynamically consistent machine-learned internal state variable approach for data-driven modeling of path-dependent materials ⋮ A Study on Affine Transformations and a Novel Universal Prediction Theory ⋮ Algebraic dynamical systems in machine learning ⋮ Toward automated hail disaster weather recognition based on spatio-temporal sequence of radar images ⋮ Option valuation under no-arbitrage constraints with neural networks ⋮ Neural network-based parameter estimation of stochastic differential equations driven by Lévy noise

Uses Software

• LSTM
• MS-COCO
• Clockwork RNN
• Evolino
• Flickr30K

Cites Work

• Supervised sequence labelling with recurrent neural networks.
• Neocognition: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
• 10.1162/153244303768966139
• Learning Deep Architectures for AI
• Training Recurrent Networks by Evolino
• Gated Orthogonal Recurrent Units: On Learning to Forget
• Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review