Phase transitions in the generalization behaviour of multilayer perceptrons: II. The influence of noise

DOI10.1088/0305-4470/30/24/017MaRDI QIDQ4255684zbMATH OpenOpenAlexFDO

Authors Bernhard Schottky, Uwe Krey

Publication date 14 February 2001

Published in Journal of Physics A: Mathematical and General (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/cond-mat/9710074

zbMATH Keywords

phase transitionsBoolean functionreplica symmetry breakinghidden unitsinput noisemultilayer perceptronstwo-layer neural networksoutput noiseinfluence of noiseGibbsian and Bayesian learninginput unitsoutput unit

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Neural networks for/in biological studies, artificial life and related topics (92B20) Dynamic and nonequilibrium phase transitions (general) in statistical mechanics (82C26) Neural nets applied to problems in time-dependent statistical mechanics (82C32)

Abstract: We extend our study of phase transitions in the generalization behaviour of multilayer perceptrons with non-overlapping receptive fields to the problem of the influence of noise, concerning e.g. the input units and/or the couplings between the input units and the hidden units of the second layer (='input noise'), or the final output unit (='output noise'). Without output noise, the output itself is given by a general, permutation-invariant Boolean function of the outputs of the hidden units. As a result we find that the phase transitions, which we found in the deterministic case, mostly persist in the presence of noise. The influence of the noise on the position of the phase transition, as well as on the behaviour in other regimes of the loading parameter $alpha$, can often be described by a simple rescaling of $alpha$ depending on strength and type of the noise. We then consider the problem of the optimal noise level for Gibbsian and Bayesian learning, looking on replica symmetry breaking as well. Finally we consider the question why learning with errors is useful at all.

Recommendations

• Generalization in a linear perceptron in the presence of noise
• Phase transitions in the generalization behaviour of multilayer neural networks
• Learning with noise in a linear perceptron
• Generalization performance of Bayes optimal classification algorithm for learning a perceptron

Cited in

(9)

• Training neural networks with structured noise improves classification and generalization
• Influence of noise on discrete time backpropagation trained networks
• On Breiman's dilemma in neural networks: phase transitions of margin dynamics
• Noisy regression and classification with continuous multilayer networks
• Weight-decay induced phase transitions in multilayer neural networks
• Perfect loss of generalization due to noise in K=2 parity machines
• Generalization in a linear perceptron in the presence of noise
• Learning with noise in a linear perceptron
• The Effect of Noise on a Class of Energy-Based Learning Rules

This page was built for publication: Phase transitions in the generalization behaviour of multilayer perceptrons: II. The influence of noise

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4255684)