Reservoir computing approaches to recurrent neural network training

DOI10.1016/j.cosrev.2009.03.005zbMath1302.68235OpenAlexW2171865010WikidataQ105583675 ScholiaQ105583675MaRDI QIDQ458488

Mantas Lukoševičius, Herbert Jaeger

Publication date: 7 October 2014

Published in: Computer Science Review (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.cosrev.2009.03.005

zbMATH Keywords

recurrent neural networksreservoir computing

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20) Research exposition (monographs, survey articles) pertaining to computer science (68-02)

Related Items (only showing first 100 items - show all)

Recurrent Kernel Machines: Computing with Infinite Echo State Networks ⋮ The echo index and multistability in input-driven recurrent neural networks ⋮ Machine-learning construction of a model for a macroscopic fluid variable using the delay-coordinate of a scalar observable ⋮ A modified batch intrinsic plasticity method for pre-training the random coefficients of extreme learning machines ⋮ Synchronization of reservoir computers with applications to communications ⋮ Unnamed Item ⋮ Search optimization, funnel topography, and dynamical criticality on the string landscape ⋮ Testing dynamical system variables for reconstruction ⋮ Regularized Variational Bayesian Learning of Echo State Networks with Delay&Sum Readout ⋮ Modeling chaotic systems: dynamical equations vs machine learning approach ⋮ Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data ⋮ The role of feedback in morphological computation with compliant bodies ⋮ Neural kernels for recursive support vector regression as a model for episodic memory ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Detection of generalized synchronization using echo state networks ⋮ Solving flows of dynamical systems by deep neural networks and a novel deep learning algorithm ⋮ Echo State Property Linked to an Input: Exploring a Fundamental Characteristic of Recurrent Neural Networks ⋮ Unstructured network topology begets order-based representation by privileged neurons ⋮ The architecture of dynamic reservoir in the echo state network ⋮ A Simple Memristive Circuit for Pattern Classification Based on Reservoir Computing ⋮ Echo state networks are universal ⋮ Time series classification with echo memory networks ⋮ Delay-dependent stability analysis of the QUAD vector field fractional order quaternion-valued memristive uncertain neutral type leaky integrator echo state neural networks ⋮ Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes ⋮ A local echo state property through the largest Lyapunov exponent ⋮ A decentralized training algorithm for echo state networks in distributed big data applications ⋮ Synthesis of recurrent neural networks for dynamical system simulation ⋮ The roles of Kerr nonlinearity in a bosonic quantum neural network ⋮ Proper choice of hyperparameters in reservoir computing of chaotic maps ⋮ A brain-inspired computational model for spatio-temporal information processing ⋮ Transfer-RLS method and transfer-FORCE learning for simple and fast training of reservoir computing models ⋮ Comparison of deep neural networks and deep hierarchical models for spatio-temporal data ⋮ Prediction of chimera in coupled map networks by means of deep learning ⋮ Unnamed Item ⋮ Re-visiting the echo state property ⋮ Optimization of the memory reset rate of a quantum echo-state network for time sequential tasks ⋮ Integration of velocity-dependent spatio-temporal structure of place cell activation during navigation in a reservoir model of prefrontal cortex ⋮ Reducing network size and improving prediction stability of reservoir computing ⋮ A reservoir-driven non-stationary hidden Markov model ⋮ Global optimization of hyper-parameters in reservoir computing ⋮ Seeking optimal parameters for achieving a lightweight reservoir computing: a computational endeavor ⋮ Dynamics of spiking map-based neural networks in problems of supervised learning ⋮ Real-time sensory-motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization ⋮ The copula echo state network ⋮ Using reservoir computer to predict and prevent extreme events ⋮ Critical neuromorphic computing based on explosive synchronization ⋮ A Biological Gradient Descent for Prediction Through a Combination of STDP and Homeostatic Plasticity ⋮ Toward Nonlinear Local Reinforcement Learning Rules Through Neuroevolution ⋮ Homeostatic Plasticity for Single Node Delay-Coupled Reservoir Computing ⋮ Timescale Separation in Recurrent Neural Networks ⋮ Symbolic Computation Using Cellular Automata-Based Hyperdimensional Computing ⋮ Nonlinear Memory Capacity of Parallel Time-Delay Reservoir Computers in the Processing of Multidimensional Signals ⋮ Assessing observability of chaotic systems using Delay Differential Analysis ⋮ Reservoir Computing with Computational Matter ⋮ Spiking networks as efficient distributed controllers ⋮ Design of a negative group delay filter via reservoir computing approach: real-time prediction of chaotic signals ⋮ A Theory of Sequence Indexing and Working Memory in Recurrent Neural Networks ⋮ Stability and memory-loss go hand-in-hand: three results in dynamics and computation ⋮ Comparison of echo state network and feed-forward neural networks in electrical load forecasting for demand response programs ⋮ DNA Reservoir Computing: A Novel Molecular Computing Approach ⋮ Deep reservoir neural networks for trees ⋮ Unnamed Item ⋮ Heuristic Dynamic Programming Using Echo State Network For Multivariable Tracking Control Of Wastewater Treatment Process ⋮ A new echo state network with variable memory length ⋮ Functional echo state network for time series classification ⋮ ECHO STATE QUEUEING NETWORKS: A COMBINATION OF RESERVOIR COMPUTING AND RANDOM NEURAL NETWORKS ⋮ Bayesian framework for simulation of dynamical systems from multidimensional data using recurrent neural network ⋮ Learning with Precise Spike Times: A New Decoding Algorithm for Liquid State Machines ⋮ Hebbian Versus Gradient Training of ESN Actors in Closed-Loop ACD ⋮ Collective dynamics of rate neurons for supervised learning in a reservoir computing system ⋮ Good and bad predictions: Assessing and improving the replication of chaotic attractors by means of reservoir computing ⋮ Evolving Carbon Nanotube Reservoir Computers ⋮ Predicting critical transitions in multiscale dynamical systems using reservoir computing ⋮ Reservoir computing with swarms ⋮ Parameter extraction with reservoir computing: Nonlinear time series analysis and application to industrial maintenance ⋮ Using machine learning to predict statistical properties of non-stationary dynamical processes: System climate,regime transitions, and the effect of stochasticity ⋮ A Receding Horizon Sliding Controller for Automotive Engine Coldstart: Design and Hardware-in-the-Loop Testing With an Echo State Network High-Fidelity Model ⋮ Direct adaptive control for nonlinear systems using a TSK fuzzy echo state network based on fractional-order learning algorithm ⋮ \(\mathrm{SO}(3)\)-invariance of informed-graph-based deep neural network for anisotropic elastoplastic materials ⋮ A multiscale multi-permeability poroplasticity model linked by recursive homogenizations and deep learning ⋮ Deep learning algorithm for data-driven simulation of noisy dynamical system ⋮ Transfer learning of chaotic systems ⋮ On explaining the surprising success of reservoir computing forecaster of chaos? The universal machine learning dynamical system with contrast to VAR and DMD ⋮ Multifunctionality in a reservoir computer ⋮ Dimension reduction in recurrent networks by canonicalization ⋮ Design Strategies for Weight Matrices of Echo State Networks ⋮ Delayed dynamical systems: networks, chimeras and reservoir computing ⋮ Detecting unstable periodic orbits based only on time series: When adaptive delayed feedback control meets reservoir computing ⋮ The reservoir’s perspective on generalized synchronization ⋮ Assessing the robustness of critical behavior in stochastic cellular automata ⋮ Unnamed Item ⋮ Stochastic nonlinear time series forecasting using time-delay reservoir computers: performance and universality ⋮ Two methods to approximate the Koopman operator with a reservoir computer ⋮ Effective connectivity at synaptic level in humans: a review and future prospects ⋮ Multidisciplinary Trends in Modern Artificial Intelligence: Turing’s Way ⋮ Learning nonlinear input-output maps with dissipative quantum systems ⋮ Stabilized neural ordinary differential equations for long-time forecasting of dynamical systems ⋮ The self-organized critical multiverse ⋮ A framework for machine learning of model error in dynamical systems

Uses Software

• darch
• Evolino

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Isolated word recognition with the Liquid State Machine: a case study
• Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning
• The cerebellum as a liquid state machine
• Fading memory and time series prediction in recurrent networks with different forms of plasticity
• Edge of chaos and prediction of computational performance for neural circuit models
• Optimization and applications of echo state networks with leaky- integrator neurons
• Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning
• Decoupled echo state networks with lateral inhibition
• An associative memory readout for ESNs with applications to dynamical pattern recognition
• An experimental unification of reservoir computing methods
• Automatic speech recognition using a predictive echo state network classifier
• Emergence of Scaling in Random Networks
• Reducing the Dimensionality of Data with Neural Networks
• Synergies Between Intrinsic and Synaptic Plasticity Mechanisms
• Reservoir optimization in recurrent neural networks using properties of Kronecker product
• Training Recurrent Networks by Evolino
• Spiking Neurons Can Learn to Solve Information Bottleneck Problems and Extract Independent Components
• Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
• Dynamical Working Memory and Timed Responses: The Role of Reverberating Loops in the Olivo-Cerebellar System
• Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks
• Collective dynamics of ‘small-world’ networks
• Neural networks and physical systems with emergent collective computational abilities.
• Analysis and Design of Echo State Networks

This page was built for publication: Reservoir computing approaches to recurrent neural network training