Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

DOI10.1016/J.NEUNET.2014.04.002zbMATH Open1347.60063arXiv1402.1613OpenAlexW1978148674WikidataQ51088425 ScholiaQ51088425MaRDI QIDQ889265FDOQ889265

Authors: Mathieu N. Galtier, Camille Marini, Gilles Wainrib, Herbert Jaeger 

Publication date: 6 November 2015

Published in: Neural Networks (Search for Journal in Brave)

Abstract: A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear Inverse Modeling (LIM), under the common principle of relative entropy minimization. The power of the new method is demonstrated on a stochastic approximation of the El Nino phenomenon studied in climate research.

Full work available at URL: https://arxiv.org/abs/1402.1613

Recommendations

• Stochastic Neural Networks With Applications to Nonlinear Time Series
• Metric entropy limits on recurrent neural network learning of linear dynamical systems
• Maximum-entropy approximations of stochastic nonlinear transductions: An extension of the Wiener theory
• scientific article; zbMATH DE number 2221002
• Modelling nonstationary dynamics

zbMATH Keywords

stochastic processesapproximationneural networksrelative entropyecho state networks[https://portal.mardi4nfdi.de/w/index.php?title=+Special%3ASearch&search=El+Ni%EF%BF%BD%EF%BF%BDo+phenomenon&go=Go El Ni��o phenomenon]linear inverse modeling

Mathematics Subject Classification ID

Probabilistic models, generic numerical methods in probability and statistics (65C20) Neural networks for/in biological studies, artificial life and related topics (92B20) Stochastic processes (60G99) Meteorology and atmospheric physics (86A10) Other physical applications of random processes (60K40)

Cites Work

• Title not available (Why is that?)
• Gaussian processes for machine learning.
• Pattern recognition and machine learning.
• On Information and Sufficiency
• Title not available (Why is that?)
• Statistical Inference for Probabilistic Functions of Finite State Markov Chains
• On Transforming a Certain Class of Stochastic Processes by Absolutely Continuous Substitution of Measures
• The Fokker-Planck equation. Methods of solutions and applications.
• Optimization and applications of echo state networks with leaky- integrator neurons
• Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
• A Fast Learning Algorithm for Deep Belief Nets
• Reservoir computing approaches to recurrent neural network training
• Calibrating volatility surfaces via relative-entropy minimization
• Error analysis of Jacobi derivative estimators for noisy signals
• A stochastic model of IndoPacific sea surface temperature anomalies
• Some Numerical Methods for Rare Events Simulation and Analysis
• A Biological Gradient Descent for Prediction Through a Combination of STDP and Homeostatic Plasticity

Cited In (3)

• A local echo state property through the largest Lyapunov exponent
• Title not available (Why is that?)
• Propagation and control of stochastic signals through universal learning networks

Uses Software

• darch
• PRMLT

This page was built for publication: Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q889265)