Learning to imitate stochastic time series in a compositional way by chaos

DOI10.1016/J.NEUNET.2009.12.006MaRDI QIDQ1784572zbMATH OpenOpenAlexWikidataFDO

Authors Jun Namikawa, Jun Tani

Publication date 27 September 2018

Published in Neural Networks (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0805.1795

zbMATH Keywords

maximum likelihood estimationmixture of expertschaosrecurrent neural network

Mathematics Subject Classification ID

Time series, auto-correlation, regression, etc. in statistics (GARCH) (62M10) Learning and adaptive systems in artificial intelligence (68T05) Strange attractors, chaotic dynamics of systems with hyperbolic behavior (37D45) Time series analysis of dynamical systems (37M10)

Abstract: This study shows that a mixture of RNN experts model can acquire the ability to generate sequences combining multiple primitive patterns by means of self-organizing chaos. By training of the model, each expert learns a primitive sequence pattern, and a gating network learns to imitate stochastic switching of the multiple primitives via a chaotic dynamics, utilizing a sensitive dependence on initial conditions. As a demonstration, we present a numerical simulation in which the model learns Markov chain switching among some Lissajous curves by a chaotic dynamics. Our analysis shows that by using a sufficient amount of training data, balanced with the network memory capacity, it is possible to satisfy the conditions for embedding the target stochastic sequences into a chaotic dynamical system. It is also shown that reconstruction of a stochastic time series by a chaotic model can be stabilized by adding a negligible amount of noise to the dynamics of the model.

Recommendations

• Evaluating generation of chaotic time series by convolutional generative adversarial networks
• Learning chaotic dynamics by neural networks
• Generating replications of chaotic time series
• Noisy time series generation by feed-forward networks
• Learning latent dynamics for partially observed chaotic systems
• scientific article; zbMATH DE number 1886443
• A dilated convolution network-based LSTM model for multi-step prediction of chaotic time series

Cites work

• scientific article; zbMATH DE number 193491 (Why is no real title available?)
• scientific article; zbMATH DE number 3025333 (Why is no real title available?)
• A model for learning to segment temporal sequences, utilizing a mixture of RNN experts together with adaptive variance
• An Introduction to Symbolic Dynamics and Coding
• Chaotic itinerancy
• Learning Nonregular Languages: A Comparison of Simple Recurrent Networks and LSTM
• Learning to generate combinatorial action sequences utilizing the initial sensitivity of deterministic dynamical systems

Cited in

(3)

• Action understanding and active inference
• Learning to generate combinatorial action sequences utilizing the initial sensitivity of deterministic dynamical systems
• A model for learning to segment temporal sequences, utilizing a mixture of RNN experts together with adaptive variance

Describes a project that uses

Uses Software

• LSTM

This page was built for publication: Learning to imitate stochastic time series in a compositional way by chaos

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1784572)