Thermodynamics of bidirectional associative memories
From MaRDI portal
Publication:6046084
Abstract: In this paper we investigate the equilibrium properties of bidirectional associative memories (BAMs). Introduced by Kosko in 1988 as a generalization of the Hopfield model to a bipartite structure, the simplest architecture is defined by two layers of neurons, with synaptic connections only between units of different layers: even without internal connections within each layer, information storage and retrieval are still possible through the reverberation of neural activities passing from one layer to another. We characterize the computational capabilities of a stochastic extension of this model in the thermodynamic limit, by applying rigorous techniques from statistical physics. A detailed picture of the phase diagram at the replica symmetric level is provided, both at finite temperature and in the noiseless regimes. Also for the latter, the critical load is further investigated up to one step of replica symmetry breaking. An analytical and numerical inspection of the transition curves (namely critical lines splitting the various modes of operation of the machine) is carried out as the control parameters - noise, load and asymmetry between the two layer sizes - are tuned. In particular, with a finite asymmetry between the two layers, it is shown how the BAM can store information more efficiently than the Hopfield model by requiring less parameters to encode a fixed number of patterns. Comparisons are made with numerical simulations of neural dynamics. Finally, a low-load analysis is carried out to explain the retrieval mechanism in the BAM by analogy with two interacting Hopfield models. A potential equivalence with two coupled Restricted Boltmzann Machines is also discussed.
Recommendations
- Analysis of Bidirectional Associative Memory using Self-consistent Signal to Noise Analysis and Statistical Neurodynamics
- Retrieval properties of bidirectional associative memories
- A bidirectional associative memory based on optimal linear associative memory
- Storing, learning and retrieving biased patterns
- Optimal bidirectional associative memories
Cites work
- scientific article; zbMATH DE number 1273988 (Why is no real title available?)
- A quantitative analysis of a generalized Hopfield model that stores and retrieves mismatched memory patterns
- A spherical Hopfield model
- Analogue neural networks on correlated random graphs
- Analytic solution of attractor neural networks on scale-free graphs
- Annealing and replica-symmetry in deep Boltzmann machines
- Bipartite Mean Field Spin Systems. Existence and Solution
- Deep Boltzmann machines: rigorous results at arbitrary depth
- Macroscopic limit of a bipartite Curie-Weiss model: a dynamical approach
- Neural networks retrieving Boolean patterns in a sea of Gaussian ones
- On the effective initialisation for restricted Boltzmann machines via duality with Hopfield model
- On the equivalence of Hopfield networks and Boltzmann machines
- Parallel retrieval of correlated patterns: from Hopfield networks to Boltzmann machines
- Reducing the Dimensionality of Data with Neural Networks
- Replica symmetry breaking in dense Hebbian neural networks
- Replica symmetry breaking in neural networks: a few steps toward rigorous results
- The Little–Hopfield model on a sparse random graph
- The adaptive interpolation method for proving replica formulas. Applications to the Curie–Weiss and Wigner spike models
- The adaptive interpolation method: a simple scheme to prove replica formulas in Bayesian inference
- The mean field Ising model trough interpolating techniques
- Thermodynamics of restricted Boltzmann machines and related learning dynamics
- Training Products of Experts by Minimizing Contrastive Divergence
- Universality in Sherrington-Kirkpatrick's spin glass model
- Universality in bipartite mean field spin glasses
This page was built for publication: Thermodynamics of bidirectional associative memories
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6046084)
