When Response Variability Increases Neural Network Robustness to Synaptic Noise

DOI10.1162/NECO.2006.18.6.1349MaRDI QIDQ5474566zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Gleb Basalyga, Emilio Salinas

Publication date 26 June 2006

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/q-bio/0510036

Mathematics Subject Classification ID

Neural networks for/in biological studies, artificial life and related topics (92B20) Neural biology (92C20)

Abstract: Cortical sensory neurons are known to be highly variable, in the sense that responses evoked by identical stimuli often change dramatically from trial to trial. The origin of this variability is uncertain, but it is usually interpreted as detrimental noise that reduces the computational accuracy of neural circuits. Here we investigate the possibility that such response variability might, in fact, be beneficial, because it may partially compensate for a decrease in accuracy due to stochastic changes in the synaptic strengths of a network. We study the interplay between two kinds of noise, response (or neuronal) noise and synaptic noise, by analyzing their joint influence on the accuracy of neural networks trained to perform various tasks. We find an interesting, generic interaction: when fluctuations in the synaptic connections are proportional to their strengths (multiplicative noise), a certain amount of response noise in the input neurons can significantly improve network performance, compared to the same network without response noise. Performance is enhanced because response noise and multiplicative synaptic noise are in some ways equivalent. These results are demonstrated analytically for the most basic network consisting of two input neurons and one output neuron performing a simple classification task, but computer simulations show that the phenomenon persists in a wide range of architectures, including recurrent (attractor) networks and sensory-motor networks that perform coordinate transformations. The results suggest that response variability could play an important dynamic role in networks that continuously learn.

Recommendations

• Synaptic input statistics tune the variability and reproducibility of neuronal responses
• Synaptic noise as a means of implementing weight-perturbation learning
• Reducing the Variability of Neural Responses: A Computational Theory of Spike-Timing-Dependent Plasticity
• Antireliability of noise-driven neurons
• Parametric amplification of signals by noise in neurons and neural networks
• SIGNAL-TO-NOISE RATIO GAIN VIA CORRELATED NOISE IN AN ENSEMBLE OF NOISY NEURONS
• Effects of the network structure and coupling strength on the noise-induced response delay of a neuronal network

Cites work

• Background Synaptic Activity as a Switch Between Dynamical States in a Network
• Stationary bumps in networks of spiking neurons

Cited in

(7)

• Impact of noise structure and network topology on tracking speed of neural networks
• Synchronization and redundancy: implications for robustness of neural learning and decision making
• Synaptic input statistics tune the variability and reproducibility of neuronal responses
• On the Noise-Enhancing Ability of Stochastic Hodgkin-Huxley Neuron Systems
• Artificial neural variability for deep learning: on overfitting, noise memorization, and catastrophic forgetting
• Heterogeneous Synaptic Weighting Improves Neural Coding in the Presence of Common Noise
• Sensitivity to Stimulus Irregularity Is Inherent in Neural Networks

This page was built for publication: When Response Variability Increases Neural Network Robustness to Synaptic Noise

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5474566)