On the overfly algorithm in deep learning of neural networks

DOI10.1016/J.AMC.2018.12.055MaRDI QIDQ2008932zbMATH OpenOpenAlexWikidataFDO

Authors Alexei Tsygvintsev

Publication date 26 November 2019

Published in Applied Mathematics and Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1807.10668

zbMATH Keywords

deep learningdynamical systemsgradient descentneural networks

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Nonconvex programming, global optimization (90C26) Approximation methods and heuristics in mathematical programming (90C59) Applications of mathematical programming (90C90) Neural networks for/in biological studies, artificial life and related topics (92B20)

Abstract: In this paper we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation.

Recommendations

• Analysis of gradient descent learning algorithms for multilayer feedforward neural networks
• Artificial Intelligence and Soft Computing - ICAISC 2004
• Dynamical Systems andOptimal Control Approach to Deep Learning
• ADAPTIVE ALGORITHMS FOR NEURAL NETWORK SUPERVISED LEARNING: A DETERMINISTIC OPTIMIZATION APPROACH
• scientific article; zbMATH DE number 1306999

Cites work

• scientific article; zbMATH DE number 4211543 (Why is no real title available?)
• scientific article; zbMATH DE number 3484411 (Why is no real title available?)
• scientific article; zbMATH DE number 2046035 (Why is no real title available?)
• Introduction to Applied Nonlinear Dynamical Systems and Chaos
• Numerical integration of differential equations in the presence of first integrals: observer method
• On the stable equilibrium points of gradient systems
• Solving Ordinary Differential Equations I
• Stability by Liapunov's direct method. With applications
• The local minima of the error surface of the \(2-2-1\) XOR network

Cited in

(3)

• A neural network model for the overfitting problem in deep learning
• Nonlinear algebra in an ACORN. With applications to deep learning
• A homotopy training algorithm for fully connected neural networks

This page was built for publication: On the overfly algorithm in deep learning of neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2008932)