Non-convex multi-species Hopfield models

DOI10.1007/S10955-018-2098-6zbMATH Open1406.82017arXiv1807.03609OpenAlexW3101329599MaRDI QIDQ1990104FDOQ1990104

Authors: Elena Agliari, Danila Migliozzi, D. Tantari 

Publication date: 24 October 2018

Published in: Journal of Statistical Physics (Search for Journal in Brave)

Abstract: In this work we introduce a multi-species generalization of the Hopfield model for associative memory, where neurons are divided into groups and both inter-groups and intra-groups pair-wise interactions are considered, with different intensities. Thus, this system contains two of the main ingredients of modern Deep neural network architectures: Hebbian interactions to store patterns of information and multiple layers coding different levels of correlations. The model is completely solvable in the low-load regime with a suitable generalization of the Hamilton-Jacobi technique, despite the Hamiltonian can be a non-definite quadratic form of the magnetizations. The family of multi-species Hopfield model includes, as special cases, the 3-layers Restricted Boltzmann Machine (RBM) with Gaussian hidden layer and the Bidirectional Associative Memory (BAM) model.

Full work available at URL: https://arxiv.org/abs/1807.03609

Recommendations

• scientific article; zbMATH DE number 1121933
• Stochastic Hopfield networks with multiplicative noise and jump Markov parameters
• Rigorous results for the Hopfield model with many patterns
• On the Hopfield neural networks and mean field theory
• Multistability in a class of stochastic delayed Hopfield neural networks
• scientific article; zbMATH DE number 837115
• Some rigorous results on the Hopfield neural network model
• Markovian jump delayed Hopfield networks with multiplicative noise
• Complex dynamics in simple Hopfield neural networks

zbMATH Keywords

neural networksBoltzmann machinesHebbian interactionsmultipartite models

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20) Kinetic theory of gases in time-dependent statistical mechanics (82C40) Neural nets applied to problems in time-dependent statistical mechanics (82C32)

Cites Work

• Deep learning
• Legendre duality of spherical and Gaussian spin glasses
• Equilibrium statistical mechanics of bipartite spin systems
• Title not available (Why is that?)
• Title not available (Why is that?)
• About a solvable mean field model of a Gaussian spin glass
• Semiconcave functions, Hamilton-Jacobi equations, and optimal control
• A logical calculus of the ideas immanent in nervous activity
• Rigorous results for the Hopfield model with many patterns
• A new stability criterion for bidirectional associative memory neural networks of neutral-type
• Multi-species mean field spin glasses. Rigorous results
• Non-convex multipartite ferromagnets
• A mechanical approach to mean field spin models
• Title not available (Why is that?)
• The free energy in a multi-species Sherrington-Kirkpatrick model
• On the equivalence of Hopfield networks and Boltzmann machines
• Matrix measure strategies for stability and synchronization of inertial BAM neural network with time delays
• Immune networks: multi-tasking capabilities at medium load
• Immune networks: multitasking capabilities near saturation
• Anergy in self-directed B lymphocytes: a statistical mechanics perspective
• Title not available (Why is that?)
• A Fast Learning Algorithm for Deep Belief Nets
• Global asymptotic stability of delayed bi-directional associative memory neural networks
• Qualitative analysis of Bi-directional Associative Memory with time delay
• The mean field Ising model trough interpolating techniques
• Finite connectivity attractor neural networks
• Multitasking attractor networks with neuronal threshold noise
• Parallel retrieval of correlated patterns: from Hopfield networks to Boltzmann machines
• The retrieval phase of the Hopfield model: A rigorous analysis of the overlap distribution
• Title not available (Why is that?)
• Rigorous results for the free energy in the Hopfield model
• Title not available (Why is that?)
• Inverse problem robustness for multi-species mean-field spin models
• The replica-symmetric solution without replica trick for the Hopfield model
• Fluctuation of the free energy in the Hopfield model
• Gibbs states of the Hopfield model with extensively many patterns.
• An Introduction to the Modeling of Neutral Networks
• Hierarchical neural networks perform both serial and parallel processing
• Replica symmetry breaking in mean-field spin glasses through the Hamilton-Jacobi technique
• Metastable states in the hierarchical Dyson model drive parallel processing in the hierarchical Hopfield network
• The replica symmetric approximation of the analogical neural network
• Exponential inequalities and convergence of moments in the replica-symmetric regime of the Hopfield model.
• About the ergodic regime in the analogical Hopfield neural networks: Moments of the partition function
• Overlap synchronisation in multipartite random energy models
• Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
• Phase Transitions and Sample Complexity in Bayes-Optimal Matrix Factorization
• High storage capacity in the Hopfield model with auto-interactions -- stability analysis
• Role of zero synapses in unsupervised feature learning

Cited In (14)

• A new mechanical approach to handle generalized Hopfield neural networks
• Nonlinear PDEs approach to statistical mechanics of dense associative memories
• Replica symmetry breaking in neural networks: a few steps toward rigorous results
• A transport equation approach for deep neural networks with quenched random weights
• Annealing and replica-symmetry in deep Boltzmann machines
• Title not available (Why is that?)
• Dense Hopfield networks in the teacher-student setting
• Propagation of chaos and large deviations in mean-field models with jumps on block-structured networks
• Memory effects of the multi-charge network model
• Learning and retrieval operational modes for three-layer restricted Boltzmann machines
• Hopfield model with planted patterns: a teacher-student self-supervised learning model
• Statistical mechanics of learning via reverberation in bidirectional associative memories
• Free energies of Boltzmann machines: self-averaging, annealed and replica symmetric approximations in the thermodynamic limit
• The relativistic Hopfield network: rigorous results

This page was built for publication: Non-convex multi-species Hopfield models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1990104)