On the overfly algorithm in deep learning of neural networks

DOI10.1016/J.AMC.2018.12.055zbMATH Open1428.90208arXiv1807.10668OpenAlexW2884614340WikidataQ128568268 ScholiaQ128568268MaRDI QIDQ2008932FDOQ2008932

Authors: Alexei Tsygvintsev 

Publication date: 26 November 2019

Published in: Applied Mathematics and Computation (Search for Journal in Brave)

Abstract: In this paper we investigate the supervised backpropagation training of multilayer neural networks from a dynamical systems point of view. We discuss some links with the qualitative theory of differential equations and introduce the overfly algorithm to tackle the local minima problem. Our approach is based on the existence of first integrals of the generalised gradient system with build-in dissipation.

Full work available at URL: https://arxiv.org/abs/1807.10668

Recommendations

• Analysis of gradient descent learning algorithms for multilayer feedforward neural networks
• Artificial Intelligence and Soft Computing - ICAISC 2004
• Dynamical Systems andOptimal Control Approach to Deep Learning
• ADAPTIVE ALGORITHMS FOR NEURAL NETWORK SUPERVISED LEARNING: A DETERMINISTIC OPTIMIZATION APPROACH
• scientific article; zbMATH DE number 1306999

zbMATH Keywords

deep learningdynamical systemsneural networksgradient descent

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Approximation methods and heuristics in mathematical programming (90C59) Applications of mathematical programming (90C90) Nonconvex programming, global optimization (90C26) Neural networks for/in biological studies, artificial life and related topics (92B20)

Cites Work

• Solving Ordinary Differential Equations I
• Title not available (Why is that?)
• Stability by Liapunov's direct method. With applications
• On the stable equilibrium points of gradient systems
• Introduction to Applied Nonlinear Dynamical Systems and Chaos
• Title not available (Why is that?)
• Title not available (Why is that?)
• The local minima of the error surface of the \(2-2-1\) XOR network
• Numerical integration of differential equations in the presence of first integrals: observer method

Cited In (3)

• A neural network model for the overfitting problem in deep learning
• Nonlinear algebra in an ACORN. With applications to deep learning
• A homotopy training algorithm for fully connected neural networks

This page was built for publication: On the overfly algorithm in deep learning of neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2008932)