A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
From MaRDI portal
Publication:5380287
Abstract: Neural network models of early sensory processing typically reduce the dimensionality of streaming input data. Such networks learn the principal subspace, in the sense of principal component analysis (PCA), by adjusting synaptic weights according to activity-dependent learning rules. When derived from a principled cost function these rules are nonlocal and hence biologically implausible. At the same time, biologically plausible local rules have been postulated rather than derived from a principled cost function. Here, to bridge this gap, we derive a biologically plausible network for subspace learning on streaming data by minimizing a principled cost function. In a departure from previous work, where cost was quantified by the representation, or reconstruction, error, we adopt a multidimensional scaling (MDS) cost function for streaming data. The resulting algorithm relies only on biologically plausible Hebbian and anti-Hebbian local learning rules. In a stochastic setting, synaptic weights converge to a stationary state which projects the input data onto the principal subspace. If the data are generated by a nonstationary distribution, the network can track the principal subspace. Thus, our result makes a step towards an algorithmic theory of neural computation.
Recommendations
- Subspace learning of neural networks
- Nonlinear data structure extraction using simple hebbian networks
- Independent component analysis by general nonlinear Hebbian-like learning rules
- A HETEROSYNAPTIC LEARNING RULE FOR NEURAL NETWORKS
- Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning
- Robust hebbian learning and noisy principal component analysis
Cites work
- A simplified neuron model as a principal component analyzer
- Dynamics of learning in linear feature-discovery networks
- scientific article; zbMATH DE number 5286897 (Why is no real title available?)
- scientific article; zbMATH DE number 1183304 (Why is no real title available?)
- scientific article; zbMATH DE number 3673370 (Why is no real title available?)
- scientific article; zbMATH DE number 1888460 (Why is no real title available?)
- scientific article; zbMATH DE number 3091264 (Why is no real title available?)
- Kernel methods and machine learning
- Multidimensional scaling. I: Theory and method
- Natural image statistics. A probabilistic approach to early computational vision.
- On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix
- On the Convergence of a Matrix Splitting Algorithm for the Symmetric Monotone Linear Complementarity Problem
- On the Convergence of the Classical Iterative Method of Solving Linear Simultaneous Equations
- Online Tracking of Linear Subspaces
- Randomized online PCA algorithms with regret bounds that are logarithmic in the dimension
- Stochastic approximation methods for constrained and unconstrained systems
Cited in
(6)- A biologically plausible neural network for multichannel canonical correlation analysis
- An efficient algorithm for sliding window based incremental principal components analysis
- Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?
- Learning overcomplete, low coherence dictionaries with linear inference
- Dimensionality-dependent generalization bounds for \(k\)-dimensional coding schemes
- Blind nonnegative source separation using biological neural networks
This page was built for publication: A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380287)
