Dynamics of Learning Near Singularities in Layered Networks

DOI10.1162/neco.2007.12-06-414zbMath1138.68048WikidataQ51899593 ScholiaQ51899593MaRDI QIDQ5453542

Jun Zhang, Tomoko Ozeki, Florent Cousseau, Haikun Wei, Shun-ichi Amari

Publication date: 2 April 2008

Published in: Neural Computation (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1162/neco.2007.12-06-414

zbMATH Keywords

radial basis function networks; hierarchical networks

Mathematics Subject Classification ID

68T05: Learning and adaptive systems in artificial intelligence

Related Items

Unnamed Item, Unnamed Item, Center Manifold Analysis of Plateau Phenomena Caused by Degeneration of Three-Layer Perceptron, Dynamics of Learning in MLP: Natural Gradient and Singularity Revisited, Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance, Natural Gradient Learning Algorithms for RBF Networks, Learning Dynamics of a Single Polar Variable Complex-Valued Neuron, Data-dependence of plateau phenomenon in learning with neural network—statistical mechanical analysis ^*, Dynamics of learning near singularities in radial basis function networks, High-dimensional dynamics of generalization error in neural networks, Noise-induced degeneration in online learning, Levenberg-Marquardt algorithm for Mackey-Glass chaotic time series prediction, How to Modify a Neural Network Gradually Without Changing Its Input-Output Functionality

Cites Work

• On the concept of attractor
• Neural theory of association and concept-formation
• Likelihood ratio of unidentifiable models and multilayer neural networks
• On-line learning of a mixture-of-experts neural network
• Algebraic Analysis for Nonidentifiable Learning Machines
• On the Problem in Model Selection of Neural Network Regression in Overrealizable Scenario
• On-line backpropagation in two-layered neural networks
• Testing in locally conic models, and application to mixture models
• Transient dynamics of on-line learning in two-layered neural networks
• Online learning dynamics of multilayer perceptrons with unidentifiable parameters
• Learning Coefficients of Layered Models When the True Distribution Mismatches the Singularities
• Learning by on-line gradient descent
• A Theory of Adaptive Pattern Classifiers
• Difficulty of Singularity in Population Coding