Role of zero synapses in unsupervised feature learning
From MaRDI portal
(Redirected from Publication:5373935)
Dynamic and nonequilibrium phase transitions (general) in statistical mechanics (82C26) Dynamic critical phenomena in statistical mechanics (82C27) Neural nets applied to problems in time-dependent statistical mechanics (82C32) Neural networks for/in biological studies, artificial life and related topics (92B20)
Abstract: Synapses in real neural circuits can take discrete values, including zero (silent or potential) synapses. The computational role of zero synapses in unsupervised feature learning of unlabeled noisy data is still unclear, thus it is important to understand how the sparseness of synaptic activity is shaped during learning and its relationship with receptive field formation. Here, we formulate this kind of sparse feature learning by a statistical mechanics approach. We find that learning decreases the fraction of zero synapses, and when the fraction decreases rapidly around a critical data size, an intrinsically structured receptive field starts to develop. Further increasing the data size refines the receptive field, while a very small fraction of zero synapses remain to act as contour detectors. This phenomenon is discovered not only in learning a handwritten digits dataset, but also in learning retinal neural activity measured in a natural-movie-stimuli experiment.
Recommendations
- Unsupervised learning and self-organization in networks of spiking neurons
- Unsupervised learning in neural computation.
- Unsupervised dynamic learning in layered neural networks
- Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
- Learning in neural networks with partially structured synaptic transitions
- Supervised and unsupervised learning with two sites of synaptic integration
- Unsupervised learnable neuron model with nonlinear interaction on dendrites
- scientific article; zbMATH DE number 1164157
Cites work
- Information, Physics, and Computation
- Noise-robust modes of the retinal population code have the geometry of ``ridges and correspond to neuronal communities
- Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapses
- Statistical Physics of Spin Glasses and Information Processing
- Why does unsupervised pre-training help deep learning?
Cited in
(3)
This page was built for publication: Role of zero synapses in unsupervised feature learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5373935)
