Lyapunov stability analysis of gradient descent-learning algorithm in network training

DOI10.5402/2011/145801MaRDI QIDQ420144zbMATH OpenOpenAlexWikidataFDO

Authors Ahmad Banakar

Publication date 21 May 2012

Published in ISRN Applied Mathematics (Search for Journal in Brave)

Copyright license Creative Commons Attribution 3.0 Unported

Full work available at URL https://doi.org/10.5402/2011/145801

zbMATH Keywords

learning algorithmLyapunov stabilitynetwork training

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Lyapunov and other classical stabilities (Lagrange, Poisson, (L^p, l^p), etc.) in control theory (93D05)

Review text

Summary: The Lyapunov stability theorem is applied to guarantee the convergence and stability of the learning algorithm for several networks. Gradient descent learning algorithm and its developed algorithms are one of the most useful learning algorithms in developing the networks. To guarantee the stability and convergence of the learning process, the upper bound of the learning rates should be investigated. Here, the Lyapunov stability theorem was developed and applied to several networks in order to guaranty the stability of the learning algorithm.

Recommendations

• A multilayer neural network. II: Stability of its learning processes
• The convergence of stochastic gradient algorithms applied to learning in neural networks
• A class of asymptotically stable algorithms for learning-rate adaptation
• Analysis of gradient descent learning algorithms for multilayer feedforward neural networks
• scientific article; zbMATH DE number 597656

Cites work

• Fuzzy identification of systems and its applications to modeling and control
• Generalized predictive control based on self-recurrent wavelet neural network for stable path tracking of mobile robots: adaptive learning rates approach
• Mechanical system modelling using recurrent neural networks via quasi- Newton learning methods

Cited in

(6)

• A class of asymptotically stable algorithms for learning-rate adaptation
• A multilayer neural network. II: Stability of its learning processes
• A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions
• On delay independent stabilization analysis for a class of switched large-scale time-delay systems
• Stabilizing and robustifying the learning mechanisms of artificial neural networks in control engineering applications
• Adaptive learning algorithm convergence in passive and reactive environments

Describes a project that uses

Uses Software

• ANFIS

This page was built for publication: Lyapunov stability analysis of gradient descent-learning algorithm in network training

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q420144)