Sparsification from dilute connectivity in a neural network model of memory (Q4240810)
From MaRDI portal
!
This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:
scientific article; zbMATH DE number 1281568
| Language | Label | Description | Also known as |
|---|---|---|---|
| default for all languages | No label defined |
||
| English | Sparsification from dilute connectivity in a neural network model of memory |
scientific article; zbMATH DE number 1281568 |
Statements
26 September 2000
0 references
hippocampus
0 references
autoassociative architecture
0 references
pattern storage and retrieval
0 references
activation thresholds
0 references
sparsification
0 references
Sparsification from dilute connectivity in a neural network model of memory (English)
0 references
0.8223216533660889
0 references
0.8161552548408508
0 references
0.7962275147438049
0 references
0.7831903100013733
0 references
It is a well-accepted theory that the hippocampus plays a key role in memory storage and pattern retrieval. The general problem of modeling hippocampus function via artificial neural networks serves several purposes, among them, to provide a computational framework for the extension and validation of various hypotheses. In 1971, David Marr proposed a theory based on the hypothesis that the archicortex is involved in short and medium-term `simple' memory. This idea has been developed further by others [e.g., by Squire and Alvarez, Gluek and Myers, Rolls and Treves]. Several authors, among them Amit and Seung, have constructed theories of memory based on neural networks with Hebbian learning and relying on the assumption that memory can be modeled via attractor dynamics. These theories tend to rely on analyses of relatively simple model networks and of their capabilities. The hope is then that relationships between network parameters (derived from brain anatomy) and measures of performance (such as storage capacity) should allow quantitative tests of the hypotheses. Along these efforts, the author of the present paper offers an autoassociative architecture for a neural network that provides pattern storage and retrieval, as a model for hippocampus function in memory. The paper introduces measures of information capacity based on the information content that the stored patterns conserve about the original ones. Further, the author explores the degree of `sparsification', or decrease in firing rate, induced by a diluted connectivity, the loss in ability to represent information, and whether this loss represents a meaningful constraint for the network's performance as a memory. Threshold regulation is used as a mechanism for controlling the firing rate, and threshold settings are used as the main tool in determining the network's operation. The sparsity or firing probability of patterns produced during retrieval is controlled through threshold manipulation, a procedure that the author argues as leading to loss of control over retrieval quality. This is partially because the network compensates for inactivation of the `active' neurons by provoking spurious firing in other neurons. Further, the author stresses the value of the ability to discriminate between neurons that should and should not be firing; and offers a detailed study of threshold variation in this context. As the author points out, this approach provides a more transparent account than the usual statistical mechanics derivations. It also makes it possible to recover the network's capacity at the same time as one derives the sparsified firing rate, as well as, the reconstruction of the original firing rate from knowledge of the network's parameters and of the sparsified firing rate. Finally, the author points out the possibility of experimental verification of his neural network models of memory. The following extract from the abstract summarizes succinctly the main results of the paper: ``We investigate an aspect of a simple, dilute network's operation that is sometimes overlooked, namely the setting of activation thresholds. An examination of several criteria for optimal threshold assignment affords several new insights. It becomes apparent that the network's capacity is insufficient to characterize the quality of its performance. We derive the degree of `sparsification' or decrease in firing probability that arises from dilution, and also the consequent losses in representational ability, and propose that they should also be taken into account. To evaluate the model's performance and suitability, we argue that one should explicitly consider the tradeoff that exists between storage of patterns and preservation of information, and its consequent constraints''.
0 references