Phase diagram and storage capacity of sequence processing neural networks

DOI10.1088/0305-4470/31/43/005zbMATH Open0962.82056arXivcond-mat/9805073OpenAlexW2022008179MaRDI QIDQ4486352FDOQ4486352

Authors: Alexander Düring, A. C. C. Coolen, David Sherrington 

Publication date: 18 June 2001

Published in: Journal of Physics A: Mathematical and General (Search for Journal in Brave)

Abstract: We solve the dynamics of Hopfield-type neural networks which store sequences of patterns, close to saturation. The asymmetry of the interaction matrix in such models leads to violation of detailed balance, ruling out an equilibrium statistical mechanical analysis. Using generating functional methods we derive exact closed equations for dynamical order parameters, viz. the sequence overlap and correlation- and response functions, in the thermodynamic limit. We calculate the time translation invariant solutions of these equations, describing stationary limit-cycles, which leads to a phase diagram. The effective retarded self-interaction usually appearing in symmetric models is here found to vanish, which causes a significantly enlarged storage capacity of $alph{a}_{c}sim0.269$, compared to $alph{a}_{c}sim0.139$ for Hopfield networks storing static patterns. Our results are tested against extensive computer simulations and excellent agreement is found.

Full work available at URL: https://arxiv.org/abs/cond-mat/9805073

Recommendations

• Cycles in symmetric sequence processing
• Storage capacity of a neural network with state-dependent synapses
• Mean-field dynamics of sequence processing neural networks with finite connectivity
• scientific article; zbMATH DE number 1121927
• Competition between pattern reconstruction and sequence processing in nonsymmetric neural networks

zbMATH Keywords

dynamics of Hopfield-type neural networks

Mathematics Subject Classification ID

Neural networks for/in biological studies, artificial life and related topics (92B20) Neural nets applied to problems in time-dependent statistical mechanics (82C32)

Cited In (13)

• Sequence Memory Based on Coherent Spin-Interaction Neural Networks
• Associative memory by recurrent neural networks with delay elements
• STORING LIMIT CYCLES USING DELAYED FEEDBACK NEURAL NETWORKS
• Dynamics of Sequence-Associative Memory for Discrete Maps in XY Spin Systems
• Mean-field dynamics of sequence processing neural networks with finite connectivity
• Onset of 'super retrieval phase' and enhancement of the storage capacity in neural networks of nonmonotonic neurons
• Symmetric sequence processing in a recurrent neural network model with a synchronous dynamics
• Spatial distribution of patterns and the Hopfield network phase space geometry
• Title not available (Why is that?)
• Increased storage capacity for hierarchically structured information in a neural network of Ising type
• Information storage matrices in neural networks
• Transient dynamics for sequence processing neural networks
• Memory Capacity for Sequences in a Recurrent Network with Biological Constraints

This page was built for publication: Phase diagram and storage capacity of sequence processing neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4486352)