Thermodynamics of bidirectional associative memories

DOI10.1088/1751-8121/ACCC60arXiv2211.09694OpenAlexW4365597485MaRDI QIDQ6046084FDOQ6046084

Authors: Adriano Barra, Giovanni Catania, Aurélien Decelle, B. Seoane 

Publication date: 15 May 2023

Published in: Journal of Physics A: Mathematical and Theoretical (Search for Journal in Brave)

Abstract: In this paper we investigate the equilibrium properties of bidirectional associative memories (BAMs). Introduced by Kosko in 1988 as a generalization of the Hopfield model to a bipartite structure, the simplest architecture is defined by two layers of neurons, with synaptic connections only between units of different layers: even without internal connections within each layer, information storage and retrieval are still possible through the reverberation of neural activities passing from one layer to another. We characterize the computational capabilities of a stochastic extension of this model in the thermodynamic limit, by applying rigorous techniques from statistical physics. A detailed picture of the phase diagram at the replica symmetric level is provided, both at finite temperature and in the noiseless regimes. Also for the latter, the critical load is further investigated up to one step of replica symmetry breaking. An analytical and numerical inspection of the transition curves (namely critical lines splitting the various modes of operation of the machine) is carried out as the control parameters - noise, load and asymmetry between the two layer sizes - are tuned. In particular, with a finite asymmetry between the two layers, it is shown how the BAM can store information more efficiently than the Hopfield model by requiring less parameters to encode a fixed number of patterns. Comparisons are made with numerical simulations of neural dynamics. Finally, a low-load analysis is carried out to explain the retrieval mechanism in the BAM by analogy with two interacting Hopfield models. A potential equivalence with two coupled Restricted Boltmzann Machines is also discussed.

Full work available at URL: https://arxiv.org/abs/2211.09694

Recommendations

• Analysis of Bidirectional Associative Memory using Self-consistent Signal to Noise Analysis and Statistical Neurodynamics
• Retrieval properties of bidirectional associative memories
• A bidirectional associative memory based on optimal linear associative memory
• Storing, learning and retrieving biased patterns
• Optimal bidirectional associative memories

zbMATH Keywords

neural networksHopfield modelassociative memoryrestricted Boltzmann machinesspin-glasses

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX) Biology and other natural sciences (92-XX)

Cites Work

• Reducing the Dimensionality of Data with Neural Networks
• Universality in bipartite mean field spin glasses
• Title not available (Why is that?)
• Universality in Sherrington-Kirkpatrick's spin glass model
• Macroscopic limit of a bipartite Curie-Weiss model: a dynamical approach
• Training Products of Experts by Minimizing Contrastive Divergence
• Bipartite Mean Field Spin Systems. Existence and Solution
• On the equivalence of Hopfield networks and Boltzmann machines
• The Little–Hopfield model on a sparse random graph
• The mean field Ising model trough interpolating techniques
• Parallel retrieval of correlated patterns: from Hopfield networks to Boltzmann machines
• Deep Boltzmann machines: rigorous results at arbitrary depth
• Annealing and replica-symmetry in deep Boltzmann machines
• A quantitative analysis of a generalized Hopfield model that stores and retrieves mismatched memory patterns
• Analytic solution of attractor neural networks on scale-free graphs
• Neural networks retrieving Boolean patterns in a sea of Gaussian ones
• Thermodynamics of restricted Boltzmann machines and related learning dynamics
• The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference
• Replica symmetry breaking in neural networks: a few steps toward rigorous results
• On the effective initialisation for restricted Boltzmann machines via duality with Hopfield model
• The adaptive interpolation method for proving replica formulas. Applications to the Curie–Weiss and Wigner spike models
• Replica symmetry breaking in dense Hebbian neural networks
• Analogue neural networks on correlated random graphs
• A spherical Hopfield model

This page was built for publication: Thermodynamics of bidirectional associative memories

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6046084)