Effect of spatial correlations on Hopfield neural network and dense associative memories

DOI10.1016/J.PHYSA.2023.128487MaRDI QIDQ2685098zbMATH OpenOpenAlexFDO

Authors Giordano De Marzo, Giulio Iannelli

Publication date 17 February 2023

Published in Physica A (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2207.05218

zbMATH Keywords

dense associative memoriesHopfield neural networksignal to noise analysis

Mathematics Subject Classification ID

Computational learning theory (68Q32) Dynamic lattice systems (kinetic Ising, etc.) and systems on graphs in time-dependent statistical mechanics (82C20) Dynamic critical phenomena in statistical mechanics (82C27) Neural nets applied to problems in time-dependent statistical mechanics (82C32) Neural biology (92C20)

Review text

The article is devoted to an investigation on the storage capacity of a Hopfield Neural Network system. One analysis the behavior of the HNN in the presence of patterns generated by a one-dimensional Ising model at inverse temperature \(\beta\) and by using a signal-to-noise technique. In the second section of the paper the signal to noise technique is introduced and is applied to Hopfield neural networks as well as DAMs (Dense Associative Memory) fed with uncorrelated Rademacher Patterns. One defines the mathematical model of the neural network, the Hamiltonian of the system and the Mattis magnetization. One shows for instance, that the neural network can store in a DAM with $p$-bodies interaction, up to O(\(N^{p-1} \)), where $N$ is the number of neurons, patterns. In the third section one investigates the behavior of a HNN in case it is fed with correlated patterns which actually mostly model real data. The Signal to noise technique is applied and analytical results are compared with numerical simulations. The results obtained in this section for Hopfield neural networks are generalized, in the fourth section, to Dense Associative Memories fed with correlated patterns. In the fifth section, some Conclusions are presented. Detailed computations for HNN are to be found in Appendix A, detailed computations for the dense associative memories can be found in Appendix B and detailed finite size scaling analysis along the transition curve for $p=2$ are presented in Appendix C.

Recommendations

• On the storage capacity of Hopfield models with correlated patterns
• Storage of sets of correlated data in neural network memories
• Properties of neural networks storing spatially correlated patterns
• Storage capacity of a neural network with state-dependent synapses
• The storage capacity of the Hopfield model and moderate deviations

Cites work

• scientific article; zbMATH DE number 2211481 (Why is no real title available?)
• On the storage capacity of Hopfield models with correlated patterns
• Programmed interactions in higher-order neural networks: Maximal capacity

Cited in

(10)

• Growth of correlation in the Hopfield model
• Generalized Guerra's interpolation schemes for dense associative neural networks
• Spatial distribution of patterns and the Hopfield network phase space geometry
• Storage of sets of correlated data in neural network memories
• scientific article; zbMATH DE number 1159248 (Why is no real title available?)
• Energy landscape of neural networks storing spatially correlated patterns
• Interpolating between boolean and extremely high noisy patterns through minimal dense associative memories
• scientific article; zbMATH DE number 5022035 (Why is no real title available?)
• Nonlinear PDEs approach to statistical mechanics of dense associative memories
• Hopfield model with planted patterns: a teacher-student self-supervised learning model

This page was built for publication: Effect of spatial correlations on Hopfield neural network and dense associative memories

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2685098)