Short-term memory capacity in networks via the restricted isometry property
From MaRDI portal
(Redirected from Publication:5383776)
Abstract: Cortical networks are hypothesized to rely on transient network activity to support short term memory (STM). In this paper we study the capacity of randomly connected recurrent linear networks for performing STM when the input signals are approximately sparse in some basis. We leverage results from compressed sensing to provide rigorous non asymptotic recovery guarantees, quantifying the impact of the input sparsity level, the input sparsity basis, and the network characteristics on the system capacity. Our analysis demonstrates that network memory capacities can scale superlinearly with the number of nodes, and in some situations can achieve STM capacities that are much larger than the network size. We provide perfect recovery guarantees for finite sequences and recovery bounds for infinite sequences. The latter analysis predicts that network STM systems may have an optimal recovery length that balances errors due to omission and recall mistakes. Furthermore, we show that the conditions yielding optimal STM capacity can be embodied in several network topologies, including networks with sparse or dense connectivities.
Recommendations
- Distributed sequence memory of multidimensional inputs in recurrent networks
- Memory Capacity for Sequences in a Recurrent Network with Biological Constraints
- Short term memory properties of sensory neural architectures
- scientific article; zbMATH DE number 2033224
- Memory and forecasting capacities of nonlinear recurrent networks
Cites work
- A network of spiking neurons for computing sparse representations in an energy-efficient way
- Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals
- Compressive sampling
- Compressive sensing and structured random matrices
- Concentration of Measure for Block Diagonal Matrices With Applications to Compressive Signal Processing
- Connectivity, dynamics, and memory in reservoir computing with binary and analog neurons
- Design strategies for weight matrices of echo state networks
- Edge of chaos and prediction of computational performance for neural circuit models
- scientific article; zbMATH DE number 49190 (Why is no real title available?)
- scientific article; zbMATH DE number 1254560 (Why is no real title available?)
- Internal representations for associative memory
- Memory in linear recurrent neural networks in continuous time
- Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
- On sparse reconstruction from Fourier and Gaussian measurements
- On the Eigenvalues of Random Matrices
- On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
- Randomly connected networks have short temporal memory
- Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations
- Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
- Sparse nonnegative solution of underdetermined linear equations by linear programming
- Suprema of chaos processes and the restricted isometry property
- Templates for convex cone problems with applications to sparse signal recovery
- The restricted isometry property for random block diagonal matrices
- Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation
Cited in
(6)- Memory and forecasting capacities of nonlinear recurrent networks
- Short term memory properties of sensory neural architectures
- Nearly extensive sequential memory lifetime achieved by coupled nonlinear neurons
- A theory of sequence indexing and working memory in recurrent neural networks
- Distributed sequence memory of multidimensional inputs in recurrent networks
- Randomly connected networks have short temporal memory
This page was built for publication: Short-term memory capacity in networks via the restricted isometry property
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5383776)
