Learning the pseudoinverse solution to network weights
From MaRDI portal
Publication:460784
DOI10.1016/J.NEUNET.2013.02.008zbMATH Open1296.68147arXiv1207.3368OpenAlexW2138061685WikidataQ44291854 ScholiaQ44291854MaRDI QIDQ460784FDOQ460784
Authors: J. Tapson, A. van Schaik 
Publication date: 9 October 2014
Published in: Neural Networks (Search for Journal in Brave)
Abstract: The last decade has seen the parallel emergence in computational neuroscience and machine learning of neural network structures which spread the input signal randomly to a higher dimensional space; perform a nonlinear activation; and then solve for a regression or classification output by means of a mathematical pseudoinverse operation. In the field of neuromorphic engineering, these methods are increasingly popular for synthesizing biologically plausible neural networks, but the "learning method" - computation of the pseudoinverse by singular value decomposition - is problematic both for biological plausibility and because it is not an online or an adaptive method. We present an online or incremental method of computing the pseudoinverse, which we argue is biologically plausible as a learning method, and which can be made adaptable for non-stationary data streams. The method is significantly more memory-efficient than the conventional computation of pseudoinverses by singular value decomposition.
Full work available at URL: https://arxiv.org/abs/1207.3368
Recommendations
- Neural Network Learning as an Inverse Problem
- Pseudo-solution of weight equations in neural networks: application for statistical parameters estimation
- Neural nets learning as an inverse problem
- Learning by a neural net in a noisy environment - the pseudo-inverse solution revisited
- Learning latent block structure in weighted networks
- Network inference with hidden units
- scientific article; zbMATH DE number 2096858
- Nonlinear weighted directed acyclic graph and a priori estimates for neural networks
- Linear constraints on weight representation for generalized learning of multilayer networks
Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20)
Cites Work
- Title not available (Why is that?)
- Self-organization and associative memory
- Generalized inverses. Theory and applications.
- Oscillation and chaos in physiological control systems
- Correlation Matrix Memories
- Some Applications of the Pseudoinverse of a Matrix
- Large-margin classification in infinite neural networks
- Title not available (Why is that?)
Cited In (8)
- STICK: spike time interval computational kernel, a framework for general purpose computation using neurons, precise timing, delays, and synchrony
- Storing cycles in Hopfield-type networks with pseudoinverse learning rule: admissibility and network topology
- Randomized algorithms for nonlinear system identification with deep learning modification
- Categorization in the pseudo-inverse neural network
- Learning by a neural net in a noisy environment - the pseudo-inverse solution revisited
- Pseudo-inverse linear discriminants for the improvement of overall classification accuracies
- An analysis of numerical issues in neural training by pseudoinversion
- Impact of random weights on nonlinear system identification using convolutional neural networks
Uses Software
This page was built for publication: Learning the pseudoinverse solution to network weights
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q460784)
