Reservoir computing approaches to recurrent neural network training

DOI10.1016/j.cosrev.2009.03.005zbMath1302.68235WikidataQ105583675 ScholiaQ105583675MaRDI QIDQ458488

Mantas Lukoševičius, Herbert Jaeger

Publication date: 7 October 2014

Published in: Computer Science Review (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.cosrev.2009.03.005

zbMATH Keywords

recurrent neural networks; reservoir computing

Mathematics Subject Classification ID

68T05: Learning and adaptive systems in artificial intelligence

92B20: Neural networks for/in biological studies, artificial life and related topics

68-02: Research exposition (monographs, survey articles) pertaining to computer science

Related Items

Unnamed Item, Testing dynamical system variables for reconstruction, Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data, Detection of generalized synchronization using echo state networks, Unnamed Item, Unnamed Item, Unnamed Item, ECHO STATE QUEUEING NETWORKS: A COMBINATION OF RESERVOIR COMPUTING AND RANDOM NEURAL NETWORKS, Collective dynamics of rate neurons for supervised learning in a reservoir computing system, Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing, Transfer learning of chaotic systems, On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to VAR and DMD, Multifunctionality in a reservoir computer, Delayed dynamical systems: networks, chimeras and reservoir computing, Unnamed Item, The self-organized critical multiverse, Unnamed Item, Search optimization, funnel topography, and dynamical criticality on the string landscape, Reducing network size and improving prediction stability of reservoir computing, Assessing observability of chaotic systems using Delay Differential Analysis, A Theory of Sequence Indexing and Working Memory in Recurrent Neural Networks, Stability and memory-loss go hand-in-hand: three results in dynamics and computation, DNA Reservoir Computing: A Novel Molecular Computing Approach, Bayesian framework for simulation of dynamical systems from multidimensional data using recurrent neural network, Learning with Precise Spike Times: A New Decoding Algorithm for Liquid State Machines, Detecting unstable periodic orbits based only on time series: When adaptive delayed feedback control meets reservoir computing, The reservoir’s perspective on generalized synchronization, Echo State Property Linked to an Input: Exploring a Fundamental Characteristic of Recurrent Neural Networks, Critical neuromorphic computing based on explosive synchronization, A Biological Gradient Descent for Prediction Through a Combination of STDP and Homeostatic Plasticity, Toward Nonlinear Local Reinforcement Learning Rules Through Neuroevolution, Homeostatic Plasticity for Single Node Delay-Coupled Reservoir Computing, Timescale Separation in Recurrent Neural Networks, Symbolic Computation Using Cellular Automata-Based Hyperdimensional Computing, Nonlinear Memory Capacity of Parallel Time-Delay Reservoir Computers in the Processing of Multidimensional Signals, Two methods to approximate the Koopman operator with a reservoir computer, The role of feedback in morphological computation with compliant bodies, A reservoir-driven non-stationary hidden Markov model, The copula echo state network, Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable, Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes, Spiking networks as efficient distributed controllers, Re-visiting the echo state property, Comparison of echo state network and feed-forward neural networks in electrical load forecasting for demand response programs, Deep reservoir neural networks for trees, Direct adaptive control for nonlinear systems using a TSK fuzzy echo state network based on fractional-order learning algorithm, Dimension reduction in recurrent networks by canonicalization, The echo index and multistability in input-driven recurrent neural networks, A modified batch intrinsic plasticity method for pre-training the random coefficients of extreme learning machines, Synchronization of reservoir computers with applications to communications, Unstructured network topology begets order-based representation by privileged neurons, Echo state networks are universal, Time series classification with echo memory networks, Delay-dependent stability analysis of the QUAD vector field fractional order quaternion-valued memristive uncertain neutral type leaky integrator echo state neural networks, Dynamics of spiking map-based neural networks in problems of supervised learning, Real-time sensory-motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization, Using reservoir computer to predict and prevent extreme events, Design of a negative group delay filter via reservoir computing approach: real-time prediction of chaotic signals, A new echo state network with variable memory length, Functional echo state network for time series classification, \(\mathrm{SO}(3)\)-invariance of informed-graph-based deep neural network for anisotropic elastoplastic materials, A multiscale multi-permeability poroplasticity model linked by recursive homogenizations and deep learning, Deep learning algorithm for data-driven simulation of noisy dynamical system, Stochastic nonlinear time series forecasting using time-delay reservoir computers: performance and universality, Effective connectivity at synaptic level in humans: a review and future prospects, A local echo state property through the largest Lyapunov exponent, A decentralized training algorithm for echo state networks in distributed big data applications, Synthesis of recurrent neural networks for dynamical system simulation, Comparison of deep neural networks and deep hierarchical models for spatio-temporal data, Heuristic Dynamic Programming Using Echo State Network For Multivariable Tracking Control Of Wastewater Treatment Process, Hebbian Versus Gradient Training of ESN Actors in Closed-Loop ACD, Evolving Carbon Nanotube Reservoir Computers, A Receding Horizon Sliding Controller for Automotive Engine Coldstart: Design and Hardware-in-the-Loop Testing With an Echo State Network High-Fidelity Model, Design Strategies for Weight Matrices of Echo State Networks, Multidisciplinary Trends in Modern Artificial Intelligence: Turing’s Way, Recurrent Kernel Machines: Computing with Infinite Echo State Networks, Regularized Variational Bayesian Learning of Echo State Networks with Delay&Sum Readout, The architecture of dynamic reservoir in the echo state network, Reservoir Computing with Computational Matter, Predicting critical transitions in multiscale dynamical systems using reservoir computing, Reservoir computing with swarms, Parameter extraction with reservoir computing: Nonlinear time series analysis and application to industrial maintenance, Using machine learning to predict statistical properties of non-stationary dynamical processes: System climate,regime transitions, and the effect of stochasticity

Uses Software

• darch
• Evolino

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Isolated word recognition with the Liquid State Machine: a case study
• Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning
• The cerebellum as a liquid state machine
• Fading memory and time series prediction in recurrent networks with different forms of plasticity
• Edge of chaos and prediction of computational performance for neural circuit models
• Optimization and applications of echo state networks with leaky- integrator neurons
• Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning
• Decoupled echo state networks with lateral inhibition
• An associative memory readout for ESNs with applications to dynamical pattern recognition
• An experimental unification of reservoir computing methods
• Automatic speech recognition using a predictive echo state network classifier
• Emergence of Scaling in Random Networks
• Reducing the Dimensionality of Data with Neural Networks
• Synergies Between Intrinsic and Synaptic Plasticity Mechanisms
• Reservoir optimization in recurrent neural networks using properties of Kronecker product
• Training Recurrent Networks by Evolino
• Spiking Neurons Can Learn to Solve Information Bottleneck Problems and Extract Independent Components
• Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
• Dynamical Working Memory and Timed Responses: The Role of Reverberating Loops in the Olivo-Cerebellar System
• Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks
• Collective dynamics of ‘small-world’ networks
• Neural networks and physical systems with emergent collective computational abilities.
• Analysis and Design of Echo State Networks