Neural networks retrieving Boolean patterns in a sea of Gaussian ones
From MaRDI portal
Abstract: Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.
Recommendations
- Storing, learning and retrieving biased patterns
- Parallel retrieval of correlated patterns: from Hopfield networks to Boltzmann machines
- On the equivalence of Hopfield networks and Boltzmann machines
- Interpolating between boolean and extremely high noisy patterns through minimal dense associative memories
- On the effective initialisation for restricted Boltzmann machines via duality with Hopfield model
Cites work
- A Fast Learning Algorithm for Deep Belief Nets
- About a solvable mean field model of a Gaussian spin glass
- About the ergodic regime in the analogical Hopfield neural networks: Moments of the partition function
- An efficient learning procedure for deep Boltzmann machines
- An Introduction to the Modeling of Neutral Networks
- Equilibrium statistical mechanics of bipartite spin systems
- Exponential inequalities and convergence of moments in the replica-symmetric regime of the Hopfield model.
- Free energy and complexity of spherical bipartite models
- Gibbs states of the Hopfield model with extensively many patterns.
- scientific article; zbMATH DE number 3876298 (Why is no real title available?)
- scientific article; zbMATH DE number 1191586 (Why is no real title available?)
- scientific article; zbMATH DE number 1273988 (Why is no real title available?)
- scientific article; zbMATH DE number 1751767 (Why is no real title available?)
- scientific article; zbMATH DE number 6276131 (Why is no real title available?)
- scientific article; zbMATH DE number 2211481 (Why is no real title available?)
- scientific article; zbMATH DE number 2212150 (Why is no real title available?)
- Metastates in the Hopfield model in the replica symmetric regime
- Multi-species mean field spin glasses. Rigorous results
- Neural networks and physical systems with emergent collective computational abilities
- Non-convex multipartite ferromagnets
- On the equivalence of Hopfield networks and Boltzmann machines
- On the stability of the quenched state in mean-field spin-glass models
- Parallel retrieval of correlated patterns: from Hopfield networks to Boltzmann machines
- Perspectives on spin glasses
- Replica symmetry breaking in mean-field spin glasses through the Hamilton-Jacobi technique
- Rigorous results for the Hopfield model with many patterns
- Rigorous solution of the Gardner problem
- Stochastic symmetry-breaking in a Gaussian Hopfield model
- The mean field Ising model trough interpolating techniques
- The replica symmetric approximation of the analogical neural network
- The retrieval phase of the Hopfield model: A rigorous analysis of the overlap distribution
- Universality in bipartite mean field spin glasses
- Universality in Sherrington-Kirkpatrick's spin glass model
Cited in
(26)- On the equivalence of Hopfield networks and Boltzmann machines
- Generalized Guerra's interpolation schemes for dense associative neural networks
- Learning and retrieval operational modes for three-layer restricted Boltzmann machines
- A new mechanical approach to handle generalized Hopfield neural networks
- Storing, learning and retrieving biased patterns
- Replica symmetry breaking in dense Hebbian neural networks
- The statistical distribution of Boolean gates in two-inputs, one-output multilayered neural networks
- A transport equation approach for deep neural networks with quenched random weights
- Interpolating between boolean and extremely high noisy patterns through minimal dense associative memories
- A quantitative analysis of a generalized Hopfield model that stores and retrieves mismatched memory patterns
- Dreaming neural networks: rigorous results
- Learning and unlearning in Hopfield-like neural network performing Boolean factor analysis
- The relativistic Hopfield network: rigorous results
- Replica symmetry breaking in neural networks: a few steps toward rigorous results
- Thermodynamics of bidirectional associative memories
- Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning
- The emergence of a concept in shallow neural networks
- On the effective initialisation for restricted Boltzmann machines via duality with Hopfield model
- Dense Hebbian neural networks: a replica symmetric picture of supervised learning
- Hopfield model with planted patterns: a teacher-student self-supervised learning model
- Statistical mechanics of learning via reverberation in bidirectional associative memories
- Dense Hopfield networks in the teacher-student setting
- The beneficial role of noises for disentanglement tasks in modular Hebbian networks
- The effect of priors on learning with restricted Boltzmann machines
- Analytical methods for continuous attractor neural networks
- Parallel retrieval of correlated patterns: from Hopfield networks to Boltzmann machines
This page was built for publication: Neural networks retrieving Boolean patterns in a sea of Gaussian ones
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1675351)
