Unsupervised learning by competing hidden units

DOI10.1073/PNAS.1820458116MaRDI QIDQ5218579zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Dmitry Krotov, J. J. Hopfield

Publication date 4 March 2020

Published in Proceedings of the National Academy of Sciences (Search for Journal in Brave)

Copyright license Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

Full work available at URL https://arxiv.org/abs/1806.10181

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Artificial neural networks and deep learning (68T07) Neural networks for/in biological studies, artificial life and related topics (92B20)

Abstract: It is widely believed that the backpropagation algorithm is essential for learning good feature detectors in early layers of artificial neural networks, so that these detectors are useful for the task performed by the higher layers of that neural network. At the same time, the traditional form of backpropagation is biologically implausible. In the present paper we propose an unusual learning rule, which has a degree of biological plausibility, and which is motivated by Hebb's idea that change of the synapse strength should be local - i.e. should depend only on the activities of the pre and post synaptic neurons. We design a learning algorithm that utilizes global inhibition in the hidden layer, and is capable of learning early feature detectors in a completely unsupervised way. These learned lower layer feature detectors can be used to train higher layer weights in a usual supervised way so that the performance of the full network is comparable to the performance of standard feedforward networks trained end-to-end with a backpropagation algorithm.

Recommendations

• Learning representations by back-propagating errors
• scientific article; zbMATH DE number 4162239
• Supervised and unsupervised learning with two sites of synaptic integration
• scientific article; zbMATH DE number 1222298
• Unsupervised dynamic learning in layered neural networks

Cited in

(7)

• Supervised and unsupervised learning with two sites of synaptic integration
• scientific article; zbMATH DE number 4162239 (Why is no real title available?)
• Contrastive similarity matching for supervised learning
• Self-organizing maps on ``what-where codes towards fully unsupervised classification
• Inferring the location of neurons within an artificial network from their activity
• Modeling and contractivity of neural-synaptic networks with Hebbian learning
• Yet another exponential Hopfield model

This page was built for publication: Unsupervised learning by competing hidden units

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5218579)