Hebbian Learning of Recurrent Connections: A Geometrical Perspective
From MaRDI portal
Abstract: We show how a Hopfield network with modifiable recurrent connections undergoing slow Hebbian learning can extract the underlying geometry of an input space. First, we use a slow/fast analysis to derive an averaged system whose dynamics derives from an energy function and therefore always converges to equilibrium points. The equilibria reflect the correlation structure of the inputs, a global object extracted through local recurrent interactions only. Second, we use numerical methods to illustrate how learning extracts the hidden geometrical structure of the inputs. Indeed, multidimensional scaling methods make it possible to project the final connectivity matrix on to a distance matrix in a high-dimensional space, with the neurons labelled by spatial position within this space. The resulting network structure turns out to be roughly convolutional. The residual of the projection defines the non-convolutional part of the connectivity which is minimized in the process. Finally, we show how restricting the dimension of the space where the neurons live gives rise to patterns similar to cortical maps. We motivate this using an energy efficiency argument based on wire length minimization. Finally, we show how this approach leads to the emergence of ocular dominance or orientation columns in primary visual cortex. In addition, we establish that the non-convolutional (or long-range) connectivity is patchy, and is co-aligned in the case of orientation learning.
Recommendations
- Hebbian learning of the statistical and geometrical structure of visual input
- Emergence of symmetric, modular, and reciprocal connections in recurrent networks with Hebbian learning
- Mathematical formulations of Hebbian learning
- Organization of receptive fields in networks with Hebbian learning: The connection between synaptic and phenomenological models
- scientific article; zbMATH DE number 2033135
- Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets
- Relating an adaptive network's structure to its emerging behaviour for Hebbian learning
- Hebbian Learning from Spiking Neural P Systems View
- Hebbian learning in parallel and modular memories
Cites work
- A simplified neuron model as a principal component analyzer
- Absolute Stability and Complete Synchronization in a Class of Neural Fields Models
- Absolute stability of global pattern formation and parallel memory storage by competitive neural networks
- Approximation of Neural Network Dynamics by Reaction-Diffusion Equations
- Dynamic properties of neural networks with adapting synapses
- Formation of topographic maps and columnar microstructures in nerve fields
- scientific article; zbMATH DE number 1983334 (Why is no real title available?)
- Kronecker products and matrix calculus in system theory
- Mathematical formulations of Hebbian learning
- Modern multidimensional scaling. Theory and applications.
- Singular perturbation methods for slow-fast dynamics
- Some Averaging and Stability Results for Random Differential Equations
- Spontaneous symmetry breaking in self-organizing neural fields
- The development of topography in the visual cortex: a review of models
- Waves, bumps, and patterns in neural field theories
Cited in
(8)- Emergence of symmetric, modular, and reciprocal connections in recurrent networks with Hebbian learning
- Stability analysis of a neural field self-organizing map
- Hebbian learning of the statistical and geometrical structure of visual input
- Continuous neural network with windowed Hebbian learning
- Informational characteristics of neural networks capable of associative learning based on Hebbian plasticity
- Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network
- Relating an adaptive network's structure to its emerging behaviour for Hebbian learning
- Modeling and contractivity of neural-synaptic networks with Hebbian learning
This page was built for publication: Hebbian Learning of Recurrent Connections: A Geometrical Perspective
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5327143)
