Echo state networks trained by Tikhonov least squares are L^2() approximators of ergodic dynamical systems
From MaRDI portal
Publication:2077652
Abstract: Echo State Networks (ESNs) are a class of single-layer recurrent neural networks with randomly generated internal weights, and a single layer of tuneable outer weights, which are usually trained by regularised linear least squares regression. Remarkably, ESNs still enjoy the universal approximation property despite the training procedure being entirely linear. In this paper, we prove that an ESN trained on a sequence of observations from an ergodic dynamical system (with invariant measure ) using Tikhonov least squares regression against a set of targets, will approximate the target function in the norm. In the special case that the targets are future observations, the ESN is learning the next step map, which allows time series forecasting. We demonstrate the theory numerically by training an ESN using Tikhonov least squares on a sequence of scalar observations of the Lorenz system.
Recommendations
Cites work
- A rigorous ODE solver and Smale's 14th problem
- Automatic speech recognition using a predictive echo state network classifier
- Data-driven reconstruction of nonlinear dynamics from sparse observation
- Deblurring Images
- Deterministic Nonperiodic Flow
- Dynamical systems with generalized hyperbolic attractors: hyperbolic, ergodic and topological properties
- Functional identification of biological neural networks using reservoir adaptation for point processes
- scientific article; zbMATH DE number 3706105 (Why is no real title available?)
- Learning grammatical structure with Echo State Networks
- Mathematical problems for the next century
- Measure theory. Vol. I and II
- Re-visiting the echo state property
- Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
- Structural stability of Lorenz attractors
- The Lorenz attractor exists
- The Lorenz attractor is mixing
- The rate of convergence in ergodic theorems
- The structure of Lorenz attractors
- Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data
Cited in
(24)- Embedding and approximation theorems for echo state networks
- Dimension reduction in recurrent networks by canonicalization
- Echo state networks are universal
- A local echo state property through the largest Lyapunov exponent
- The asymptotic performance of linear echo state neural networks
- Symmetry kills the square in a multifunctional reservoir computer
- Reservoir computing with an inertial form
- scientific article; zbMATH DE number 6453355 (Why is no real title available?)
- Analysis and Design of Echo State Networks
- Predicting turbulent dynamics with the convolutional autoencoder echo state network
- Fading memory echo state networks are universal
- Approximation bounds for random neural networks and reservoir systems
- Generalised synchronisations, embeddings, and approximations for continuous time reservoir computers
- Learning Theory for Dynamical Systems
- Learning strange attractors with reservoir systems
- Data-informed reservoir computing for efficient time-series prediction
- Reducing echo state network size with controllability matrices
- Data-driven cold starting of good reservoirs
- On the emergence of numerical instabilities in next generation reservoir computing
- Generic and isometric embeddings in reservoir computers
- Minimal deterministic echo state networks outperform random reservoirs in learning chaotic dynamics
- Forecasting causal dynamics with universal reservoirs
- Limits of learning dynamical systems
- A mathematical mechanism of reservoir computing for dynamical systems
This page was built for publication: Echo state networks trained by Tikhonov least squares are \(L^2(\mu)\) approximators of ergodic dynamical systems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2077652)
