Fundamental aspects of noise in analog-hardware neural networks

DOI10.1063/1.5120824MaRDI QIDQ4973007zbMATH OpenOpenAlexWikidataFDO

Authors

Publication date 29 November 2019

Published in Chaos: An Interdisciplinary Journal of Nonlinear Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1907.09002

Mathematics Subject Classification ID

Neural networks for/in biological studies, artificial life and related topics (92B20)

Abstract: We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multi-layer networks. The main focus of our study are neural networks in analogue hardware, yet the methodology provides insight for networks in general. The system under study consists of noisy linear nodes, and we investigate the signal-to-noise ratio at the network's outputs which is the upper limit to such a system's computing accuracy. We consider additive and multiplicative noise which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks and noise amplitudes were obtained from a physically implemented recurrent neural network and therefore correspond to a real-world system. Analytic solutions agree exceptionally well with numerical data, enabling clear identification of the most critical components and aspects for noise management. Focusing on linear nodes isolates the impact of network connections and allows us to derive strategies for mitigating noise. Our work is the starting point in addressing this aspect of analogue neural networks, and our results identify notoriously sensitive points while simultaneously highlighting the robustness of such computational systems.

Recommendations

• Analog computation via neural networks
• scientific article; zbMATH DE number 1989742
• Sensitivity Measurement of Neural Hardware: A Simulation Based Study
• scientific article; zbMATH DE number 1041088
• Influence of noise on the function of a physiological neural network

Cites work

• A logical calculus of the ideas immanent in nervous activity
• All-optical machine learning using diffractive deep neural networks
• Neural networks and physical systems with emergent collective computational abilities

Cited in

(6)

• An analysis of noisy RAM and neural nets
• HEMT noise neural model based on bias conditions
• Noise Impact on a Recurrent Neural Network with a Linear Activation Function
• Time shifts to reduce the size of reservoir computers
• Multiplexing-based control of stochastic resonance
• Excitable response of a noisy adaptive network of spiking lasers

This page was built for publication: Fundamental aspects of noise in analog-hardware neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4973007)