Learning Topology and Dynamics of Large Recurrent Neural Networks

DOI10.1109/TSP.2014.2358956zbMATH Open1393.92005arXiv1410.1174OpenAlexW2091877371MaRDI QIDQ4579571FDOQ4579571

Authors: Yiyuan She, Yuejia He, Dapeng Wu 

Publication date: 22 August 2018

Published in: IEEE Transactions on Signal Processing (Search for Journal in Brave)

Abstract: Large-scale recurrent networks have drawn increasing attention recently because of their capabilities in modeling a large variety of real-world phenomena and physical mechanisms. This paper studies how to identify all authentic connections and estimate system parameters of a recurrent network, given a sequence of node observations. This task becomes extremely challenging in modern network applications, because the available observations are usually very noisy and limited, and the associated dynamical system is strongly nonlinear. By formulating the problem as multivariate sparse sigmoidal regression, we develop simple-to-implement network learning algorithms, with rigorous convergence guarantee in theory, for a variety of sparsity-promoting penalty forms. A quantile variant of progressive recurrent network screening is proposed for efficient computation and allows for direct cardinality control of network topology in estimation. Moreover, we investigate recurrent network stability conditions in Lyapunov's sense, and integrate such stability constraints into sparse network learning. Experiments show excellent performance of the proposed algorithms in network topology identification and forecasting.

Full work available at URL: https://arxiv.org/abs/1410.1174

Mathematics Subject Classification ID

Neural networks for/in biological studies, artificial life and related topics (92B20)

Cited In (3)

• Riemannian metrics for neural networks II: recurrent networks and learning symbolic data sequences
• Resonant spatiotemporal learning in large random recurrent networks
• Analysis of generalized Bregman surrogate algorithms for nonsmooth nonconvex statistical learning

This page was built for publication: Learning Topology and Dynamics of Large Recurrent Neural Networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4579571)