Dynamics of Learning Near Singularities in Layered Networks

DOI10.1162/neco.2007.12-06-414zbMath1138.68048OpenAlexW2125911849WikidataQ51899593 ScholiaQ51899593MaRDI QIDQ5453542

Jun Zhang, Tomoko Ozeki, Florent Cousseau, Haikun Wei, Shun-ichi Amari

Publication date: 2 April 2008

Published in: Neural Computation (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1162/neco.2007.12-06-414

zbMATH Keywords

radial basis function networkshierarchical networks

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Related Items (14)

Statistical mechanical analysis of learning dynamics of two-layer perceptron with multiple output units ⋮ Dynamics of learning near singularities in radial basis function networks ⋮ Center Manifold Analysis of Plateau Phenomena Caused by Degeneration of Three-Layer Perceptron ⋮ Natural Gradient Learning Algorithms for RBF Networks ⋮ Learning Dynamics of a Single Polar Variable Complex-Valued Neuron ⋮ Dynamics of Learning in MLP: Natural Gradient and Singularity Revisited ⋮ How to Modify a Neural Network Gradually Without Changing Its Input-Output Functionality ⋮ Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance ⋮ Unnamed Item ⋮ High-dimensional dynamics of generalization error in neural networks ⋮ Levenberg-Marquardt algorithm for Mackey-Glass chaotic time series prediction ⋮ Noise-induced degeneration in online learning ⋮ Data-dependence of plateau phenomenon in learning with neural network—statistical mechanical analysis ^* ⋮ Unnamed Item

Cites Work

• On the concept of attractor
• Neural theory of association and concept-formation
• Likelihood ratio of unidentifiable models and multilayer neural networks
• On-line learning of a mixture-of-experts neural network
• Algebraic Analysis for Nonidentifiable Learning Machines
• On the Problem in Model Selection of Neural Network Regression in Overrealizable Scenario
• On-line backpropagation in two-layered neural networks
• Testing in locally conic models, and application to mixture models
• Transient dynamics of on-line learning in two-layered neural networks
• Online learning dynamics of multilayer perceptrons with unidentifiable parameters
• Learning Coefficients of Layered Models When the True Distribution Mismatches the Singularities
• Learning by on-line gradient descent
• A Theory of Adaptive Pattern Classifiers
• Difficulty of Singularity in Population Coding

This page was built for publication: Dynamics of Learning Near Singularities in Layered Networks