Learning the pseudoinverse solution to network weights
From MaRDI portal
Abstract: The last decade has seen the parallel emergence in computational neuroscience and machine learning of neural network structures which spread the input signal randomly to a higher dimensional space; perform a nonlinear activation; and then solve for a regression or classification output by means of a mathematical pseudoinverse operation. In the field of neuromorphic engineering, these methods are increasingly popular for synthesizing biologically plausible neural networks, but the "learning method" - computation of the pseudoinverse by singular value decomposition - is problematic both for biological plausibility and because it is not an online or an adaptive method. We present an online or incremental method of computing the pseudoinverse, which we argue is biologically plausible as a learning method, and which can be made adaptable for non-stationary data streams. The method is significantly more memory-efficient than the conventional computation of pseudoinverses by singular value decomposition.
Recommendations
- Neural Network Learning as an Inverse Problem
- Pseudo-solution of weight equations in neural networks: application for statistical parameters estimation
- Neural nets learning as an inverse problem
- Learning by a neural net in a noisy environment - the pseudo-inverse solution revisited
- Learning latent block structure in weighted networks
- Network inference with hidden units
- scientific article; zbMATH DE number 2096858
- Nonlinear weighted directed acyclic graph and a priori estimates for neural networks
- Linear constraints on weight representation for generalized learning of multilayer networks
Cites work
- scientific article; zbMATH DE number 3652509 (Why is no real title available?)
- scientific article; zbMATH DE number 3111121 (Why is no real title available?)
- Correlation Matrix Memories
- Generalized inverses. Theory and applications.
- Large-margin classification in infinite neural networks
- Oscillation and chaos in physiological control systems
- Self-organization and associative memory
- Some Applications of the Pseudoinverse of a Matrix
Cited in
(9)- Learning by a neural net in a noisy environment - the pseudo-inverse solution revisited
- Categorization in the pseudo-inverse neural network
- Pseudo inverse versus iterated projection: novel learning approach and its application on broad learning system
- Impact of random weights on nonlinear system identification using convolutional neural networks
- Storing cycles in Hopfield-type networks with pseudoinverse learning rule: admissibility and network topology
- STICK: spike time interval computational kernel, a framework for general purpose computation using neurons, precise timing, delays, and synchrony
- Pseudo-inverse linear discriminants for the improvement of overall classification accuracies
- Randomized algorithms for nonlinear system identification with deep learning modification
- An analysis of numerical issues in neural training by pseudoinversion
This page was built for publication: Learning the pseudoinverse solution to network weights
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q460784)
