Dynamics of Learning in MLP: Natural Gradient and Singularity Revisited

DOI10.1162/neco_a_01029zbMath1472.68128OpenAlexW2766357725WikidataQ48003393 ScholiaQ48003393MaRDI QIDQ5157104

Tomoko Ozeki, Masato Okada, Ryo Karakida, Yuki Yoshida, Shun-ichi Amari

Publication date: 12 October 2021

Published in: Neural Computation (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1162/neco_a_01029

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Learning and adaptive systems in artificial intelligence (68T05)

Related Items (3)

Statistical mechanical analysis of learning dynamics of two-layer perceptron with multiple output units ⋮ Center Manifold Analysis of Plateau Phenomena Caused by Degeneration of Three-Layer Perceptron ⋮ Noise-induced degeneration in online learning

Uses Software

• darch

Cites Work

• Unnamed Item
• Information geometry and its applications
• Neocognition: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
• Reducing the Dimensionality of Data with Neural Networks
• Algebraic Geometry and Statistical Learning Theory
• Learning Deep Architectures for AI
• A Steepest-Ascent Method for Solving Optimum Programming Problems
• Riemannian metrics for neural networks I: feedforward networks
• Riemannian metrics for neural networks II: recurrent networks and learning symbolic data sequences
• Unified framework for information integration based on information geometry
• Learning representations by back-propagating errors
• Dynamics of Learning Near Singularities in Layered Networks
• Singularities Affect Dynamics of Learning in Neuromanifolds
• A Fast Learning Algorithm for Deep Belief Nets
• New Classes of Synchronous Codes
• A Stochastic Approximation Method

This page was built for publication: Dynamics of Learning in MLP: Natural Gradient and Singularity Revisited