On the storage capacity of Hopfield models with correlated patterns (Q1296722): Difference between revisions

It is well known that the Hopfield model of a fully connected neural network with \(N\) neurons with the original Hebbian learning rule can store a number of patters equal to \(C N/\ln N\) without error, and equal to \(\alpha_c N\) (with \(\alpha_c\approx 0.138\) according to numerical and replica results) if a small fraction of errors is tolerated. These results hold with probability tending to one rapidly if the patterns are assumed to be i.i.d. random variables with each component a symmetric i.i.d. Rademacher random variable. The present paper presents a rigorous investigation or the question what becomes of these results if the patterns are correlated, either ``semantically'' or ``spatially''. More specifically, it is supposed that either for any \(i\), \(\xi^\mu_i\), \(\mu\in {\mathbb N}\), form a Markov chain with state space \(\{-1,+1\}\), which are in turn independent for different \(i\), or vice versa that for each \(\mu\), \(\xi^\mu_i,i\in {\mathbb N}\), is an family of independent Markov chains. The results obtained are the following: 1) If perfect storage is required, the storage capacity decreases linearly the degree of correlation increases in the semantic case, while the opposite is true in the case of spatial correlation. 2) If storage with error is tolerated, the bounds obtained for \(a_c\) decrease with the correlation in both cases.