Echo state networks are universal
From MaRDI portal
Publication:2182904
Abstract: This paper shows that echo state networks are universal uniform approximants in the context of discrete-time fading memory filters with uniformly bounded inputs defined on negative infinite times. This result guarantees that any fading memory input/output system in discrete time can be realized as a simple finite-dimensional neural network-type state-space model with a static linear readout map. This approximation is valid for infinite time intervals. The proof of this statement is based on fundamental results, also presented in this work, about the topological nature of the fading memory property and about reservoir computing systems generated by continuous reservoir maps.
Recommendations
- Fading memory echo state networks are universal
- Re-visiting the echo state property
- Echo state networks trained by Tikhonov least squares are \(L^2(\mu)\) approximators of ergodic dynamical systems
- Embedding and approximation theorems for echo state networks
- Echo state networks with filter neurons and a delay\&sum readout
Cites work
- A fixed-point farrago
- A local echo state property through the largest Lyapunov exponent
- A Representation Theorem for Continuous Functions of Several Variables
- An experimental unification of reservoir computing methods
- Approximating nonlinear fading-memory operators using neural network models
- Approximation by superpositions of a sigmoidal function
- Closedness of sum spaces and the generalized Schrödinger problem
- Dynamical systems
- Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks
- ESTIMATING THE APPROXIMATION ERROR IN LEARNING THEORY
- Expressive power of first-order recurrent neural networks determined by their attractor dynamics
- Fading memory and the problem of approximating nonlinear operators with Volterra series
- scientific article; zbMATH DE number 3117676 (Why is no real title available?)
- scientific article; zbMATH DE number 3147699 (Why is no real title available?)
- scientific article; zbMATH DE number 3172146 (Why is no real title available?)
- scientific article; zbMATH DE number 6125590 (Why is no real title available?)
- scientific article; zbMATH DE number 3195732 (Why is no real title available?)
- Learning Theory
- Liquid state machines: motivation, theory, and applications
- Multilayer feedforward networks are universal approximators
- Notes on fading-memory conditions
- On the mathematical foundations of learning
- Re-visiting the echo state property
- Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
- Reservoir computing approaches to recurrent neural network training
- Structure theorems for nonlinear systems
- The dynamic universality of sigmoidal neural networks
- Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems
Cited in
(66)- Re-visiting the echo state property
- Embedding and approximation theorems for echo state networks
- Dimension reduction in recurrent networks by canonicalization
- Echo state networks trained by Tikhonov least squares are \(L^2(\mu)\) approximators of ergodic dynamical systems
- Learning nonlinear input-output maps with dissipative quantum systems
- Memory and forecasting capacities of nonlinear recurrent networks
- The echo index and multistability in input-driven recurrent neural networks
- Modeling chaotic systems: dynamical equations vs machine learning approach
- Neural kernels for recursive support vector regression as a model for episodic memory
- Data-driven reconstruction of nonlinear dynamics from sparse observation
- Echo state network-based online optimal control for discrete-time nonlinear systems
- The asymptotic performance of linear echo state neural networks
- Predicting critical transitions in multiscale dynamical systems using reservoir computing
- Breaking symmetries of the reservoir equations in echo state networks
- Parameter extraction with reservoir computing: nonlinear time series analysis and application to industrial maintenance
- Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systems
- Dynamical systems as temporal feature spaces
- Understanding recurrent neural networks using nonequilibrium response theory
- Symmetry kills the square in a multifunctional reservoir computer
- Embedding information onto a dynamical system
- Real-time thermoacoustic data assimilation
- scientific article; zbMATH DE number 7306919 (Why is no real title available?)
- Stability and memory-loss go hand-in-hand: three results in dynamics and computation
- scientific article; zbMATH DE number 7164779 (Why is no real title available?)
- Echo state property linked to an input: exploring a fundamental characteristic of recurrent neural networks
- Predicting turbulent dynamics with the convolutional autoencoder echo state network
- A framework for machine learning of model error in dynamical systems
- Fading memory echo state networks are universal
- Robust optimization and validation of echo state networks for learning chaotic dynamics
- Transport in reservoir computing
- Approximation bounds for random neural networks and reservoir systems
- Generalised synchronisations, embeddings, and approximations for continuous time reservoir computers
- Inferring unknown unknowns: regularized bias-aware ensemble Kalman filter
- Stability analysis of chaotic systems from data
- Learning strange attractors with reservoir systems
- Designing universal causal deep learning models: The geometric (Hyper)transformer
- Learn to synchronize, synchronize to learn
- Reducing echo state network size with controllability matrices
- Transitions in echo index and dependence on input repetitions
- Data-driven cold starting of good reservoirs
- A real-time digital twin of azimuthal thermoacoustic instabilities
- Mini-workshop: Mathematics of entropic AI in the natural sciences. Abstracts from the mini-workshop held April 7--12, 2024
- Constrained dynamics, stochastic numerical methods and the modeling of complex systems. Abstracts from the workshop held May 26--31, 2024
- Generic and isometric embeddings in reservoir computers
- Signature methods in machine learning
- Universal approximation of dynamical systems by semiautonomous neural ODEs and applications
- New analytic formulas for memory and prediction functions in reservoir computers with time delays
- Signature methods in stochastic portfolio theory
- On the choice of the non-trainable internal weights in random feature maps for forecasting chaotic dynamical systems
- Minimal deterministic echo state networks outperform random reservoirs in learning chaotic dynamics
- How more data can hurt: instability and regularization in next-generation reservoir computing
- A brief survey on the approximation theory for sequence modelling
- Refined Kolmogorov complexity of analog, evolving and stochastic recurrent neural networks
- Universality of reservoir systems with recurrent neural networks
- Forecasting causal dynamics with universal reservoirs
- Full error analysis of the random deep splitting method for nonlinear parabolic PDEs and PIDEs
- Small-controllability of state-affine nonlinear systems
- Next-generation reservoir computing for dynamical inference
- Global universal approximation of functional input maps on weighted spaces
- Synchronizing pretrained kernel regressors with applications to American option pricing
- A finite-sample generalization bound for stable LPV systems
- Neural networks in non-metric spaces
- A mathematical mechanism of reservoir computing for dynamical systems
- Doubly stochastic inter-assembly coupling via entropic optimal transport in echo-state networks for chaotic flows
- Reservoir computing with the Kuramoto model
- Advanced Data Augmentation Methods in Neural Network for Functional Data Classification
This page was built for publication: Echo state networks are universal
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2182904)
