Contrastive similarity matching for supervised learning
From MaRDI portal
Abstract: We propose a novel biologically-plausible solution to the credit assignment problem motivated by observations in the ventral visual pathway and trained deep neural networks. In both, representations of objects in the same category become progressively more similar, while objects belonging to different categories become less similar. We use this observation to motivate a layer-specific learning goal in a deep network: each layer aims to learn a representational similarity matrix that interpolates between previous and later layers. We formulate this idea using a contrastive similarity matching objective function and derive from it deep neural networks with feedforward, lateral, and feedback connections, and neurons that exhibit biologically-plausible Hebbian and anti-Hebbian plasticity. Contrastive similarity matching can be interpreted as an energy-based learning algorithm, but with significant differences from others in how a contrastive function is constructed.
Recommendations
- Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?
- Unsupervised learning by competing hidden units
- Neurally plausible mechanisms for learning selective and invariant representations
- Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network
- Unsupervised learning of invariant representations
Cites work
- An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity
- Blind nonnegative source separation using biological neural networks
- Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network
- scientific article; zbMATH DE number 4074520 (Why is no real title available?)
- Learning representations by back-propagating errors
- Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?
This page was built for publication: Contrastive similarity matching for supervised learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5004354)
