Memory Capacity of Neural Networks with Threshold and Rectified Linear Unit Activations

DOI10.1137/20M1314884MaRDI QIDQ5037553zbMATH OpenOpenAlexFDO

Authors Roman Vershynin

Publication date 1 March 2022

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1137/20m1314884

zbMATH Keywords

neural networksmultilayer perceptronmemory capacity

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Artificial neural networks and deep learning (68T07) Neural networks for/in biological studies, artificial life and related topics (92B20)

Recommendations

• On the capacity of associative memories with linear threshold functions
• The capacity of feedforward neural networks
• scientific article; zbMATH DE number 2202162
• Long term memory storage capacity of multiconnected neural networks
• Neural networks with memory
• Storage capacity of a neural network with state-dependent synapses
• On neural network kernels and the storage capacity problem

Cites work

• scientific article; zbMATH DE number 1740985 (Why is no real title available?)
• scientific article; zbMATH DE number 7064043 (Why is no real title available?)
• Bounds on the learning capacity of some multi-layer networks
• Enumeration of Seven-Argument Threshold Functions
• Gradient descent optimizes over-parameterized deep ReLU networks
• High-dimensional probability. An introduction with applications in data science
• On the capabilities of multilayer perceptrons
• The capacity of feedforward neural networks

Cited in

(11)

• The capacity of feedforward neural networks
• Long term memory storage capacity of multiconnected neural networks
• Expressive power of ReLU and step networks under floating-point operations
• Just least squares: binary compressive sampling with low generative intrinsic dimension
• Approximation in shift-invariant spaces with deep ReLU neural networks
• Information theory and recovery algorithms for data fusion in Earth observation
• Designing universal causal deep learning models: The geometric (Hyper)transformer
• Stable recovery of entangled weights: towards robust identification of deep neural networks from minimal samples
• The interpolation phase transition in neural networks: memorization and generalization under lazy training
• scientific article; zbMATH DE number 7370609 (Why is no real title available?)
• Overparameterized neural networks implement associative memory

This page was built for publication: Memory Capacity of Neural Networks with Threshold and Rectified Linear Unit Activations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5037553)