Phase transitions in the generalization behaviour of multilayer perceptrons: II. The influence of noise

DOI10.1088/0305-4470/30/24/017zbMATH Open0954.82021arXivcond-mat/9710074OpenAlexW1970613237MaRDI QIDQ4255684FDOQ4255684

Authors: Bernhard Schottky, Uwe Krey 

Publication date: 14 February 2001

Published in: Journal of Physics A: Mathematical and General (Search for Journal in Brave)

Abstract: We extend our study of phase transitions in the generalization behaviour of multilayer perceptrons with non-overlapping receptive fields to the problem of the influence of noise, concerning e.g. the input units and/or the couplings between the input units and the hidden units of the second layer (='input noise'), or the final output unit (='output noise'). Without output noise, the output itself is given by a general, permutation-invariant Boolean function of the outputs of the hidden units. As a result we find that the phase transitions, which we found in the deterministic case, mostly persist in the presence of noise. The influence of the noise on the position of the phase transition, as well as on the behaviour in other regimes of the loading parameter $alpha$, can often be described by a simple rescaling of $alpha$ depending on strength and type of the noise. We then consider the problem of the optimal noise level for Gibbsian and Bayesian learning, looking on replica symmetry breaking as well. Finally we consider the question why learning with errors is useful at all.

Full work available at URL: https://arxiv.org/abs/cond-mat/9710074

Recommendations

• Generalization in a linear perceptron in the presence of noise
• Phase transitions in the generalization behaviour of multilayer neural networks
• Learning with noise in a linear perceptron
• Generalization performance of Bayes optimal classification algorithm for learning a perceptron

zbMATH Keywords

phase transitionsBoolean functionreplica symmetry breakinghidden unitsinput noisemultilayer perceptronstwo-layer neural networksoutput noiseinfluence of noiseGibbsian and Bayesian learninginput unitsoutput unit

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20) Dynamic and nonequilibrium phase transitions (general) in statistical mechanics (82C26) Neural nets applied to problems in time-dependent statistical mechanics (82C32)

Cited In (8)

• Weight-decay induced phase transitions in multilayer neural networks
• The Effect of Noise on a Class of Energy-Based Learning Rules
• Perfect loss of generalization due to noise in K=2 parity machines
• Training neural networks with structured noise improves classification and generalization
• Influence of noise on discrete time backpropagation trained networks
• Generalization in a linear perceptron in the presence of noise
• Learning with noise in a linear perceptron
• On Breiman's dilemma in neural networks: phase transitions of margin dynamics

This page was built for publication: Phase transitions in the generalization behaviour of multilayer perceptrons: II. The influence of noise

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4255684)