The echo index and multistability in input-driven recurrent neural networks

DOI10.1016/J.PHYSD.2020.132609zbMATH Open1484.68181arXiv2001.07694OpenAlexW3001462218MaRDI QIDQ2127403FDOQ2127403

Authors: Andrea Ceni, Peter Ashwin, Lorenzo Livi, Claire Postlethwaite 

Publication date: 20 April 2022

Published in: Physica D (Search for Journal in Brave)

Abstract: A recurrent neural network (RNN) possesses the echo state property (ESP) if, for a given input sequence, it ``forgets any internal states of the driven (nonautonomous) system and asymptotically follows a unique, possibly complex trajectory. The lack of ESP is conventionally understood as a lack of reliable behaviour in RNNs. Here, we show that RNNs can reliably perform computations under a more general principle that accounts only for their local behaviour in phase space. To this end, we formulate a generalisation of the ESP and introduce an echo index to characterise the number of simultaneously stable responses of a driven RNN. We show that it is possible for the echo index to change with inputs, highlighting a potential source of computational errors in RNNs due to characteristics of the inputs driving the dynamics.

Full work available at URL: https://arxiv.org/abs/2001.07694

Recommendations

• Multistability in Recurrent Neural Networks
• Delay-dependent multistability in recurrent neural networks
• Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks
• Multistability and instability analysis of recurrent neural networks with time-varying delays
• Delay-independent exponential stability of recurrent neural networks
• Multistability of delayed recurrent neural networks with Mexican hat activation functions
• Dynamical behavior of recurrent neural networks with different external inputs

zbMATH Keywords

machine learningnonautonomous dynamical systemsmultistabilityinput-driven systemsrecurrent neural networksecho state property

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Topological dynamics of nonautonomous systems (37B55) Networks and circuits as models of computation; circuit complexity (68Q06)

Cites Work

• Title not available (Why is that?)
• Morse Decomposition of Attractors for Non-autonomous Dynamical Systems
• Nonautonomous continuation of bounded solutions
• Random fixed point theorems with an application to random differential equations in Banach spaces
• Title not available (Why is that?)
• Forward attraction in nonautonomous difference equations
• Optimization and applications of echo state networks with leaky- integrator neurons
• TOWARDS A MORSE THEORY FOR RANDOM DYNAMICAL SYSTEMS
• Reservoir computing approaches to recurrent neural network training
• Limitations of pullback attractors for processes
• Detection of generalized synchronization using echo state networks
• Re-visiting the echo state property
• Echo state networks are universal
• Title not available (Why is that?)
• Echo State Property Linked to an Input: Exploring a Fundamental Characteristic of Recurrent Neural Networks
• A Geometrical Analysis of Global Stability in Trained Feedback Networks

Cited In (6)

• Learn to synchronize, synchronize to learn
• Stimulus-Driven and Spontaneous Dynamics in Excitatory-Inhibitory Recurrent Neural Networks for Sequence Representation
• Transitions in echo index and dependence on input repetitions
• Embedding information onto a dynamical system
• Minimal model for reservoir computing
• Generalised synchronisations, embeddings, and approximations for continuous time reservoir computers

This page was built for publication: The echo index and multistability in input-driven recurrent neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2127403)