The gradient descent method from the perspective of fractional calculus

DOI10.1002/MMA.7127MaRDI QIDQ5011114zbMATH OpenOpenAlexFDO

Authors Pham Viet Hai, Joel A. Rosenfeld

Publication date 27 August 2021

Published in Mathematical Methods in the Applied Sciences (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1911.08058

zbMATH Keywords

fractional calculusgradient methodoptimization

Mathematics Subject Classification ID

Fractional derivatives and integrals (26A33) Fractional ordinary differential equations (34A08) Numerical optimization and variational techniques (65K10) Convex programming (90C25)

Abstract: Motivated by gradient methods in optimization theory, we give methods based on $psi$-fractional derivatives of order $alpha$ in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail. This paper also presents an Adams-Bashforth-Moulton (ABM) method for the estimation of solutions to equations involving $psi$-fractional derivatives. Numerical examples using the ABM method show that the fractional order $alpha$ and weight $psi$ are tunable parameters, which can be helpful for improving the performance of gradient descent methods.

Recommendations

• Generalization of the gradient method with fractional order gradient direction
• Study on fractional order gradient methods
• A quasi fractional order gradient descent method with adaptive stepsize and its application in system identification
• Fractional-order gradient descent learning of BP neural networks with Caputo derivative
• Applications of fractional gradient descent method with adaptive momentum in BP neural networks

Cited in

(14)

• An efficient gradient forecasting search method utilizing the discrete difference equation prediction model
• Study on fractional order gradient methods
• Fractional-order gradient descent learning of BP neural networks with Caputo derivative
• Fractional differential equation approach for convex optimization with convergence rate analysis
• Generalization of the gradient method with fractional order gradient direction
• A quasi fractional order gradient descent method with adaptive stepsize and its application in system identification
• Applications of fractional gradient descent method with adaptive momentum in BP neural networks
• Function approximation using a discrete fractional order gradient descent law
• Accelerating gradient descent and Adam via fractional gradients
• Fractional Order Learning Methods for Nonlinear System Identification Based on Fuzzy Neural Network
• Fractional order iteration for gradient descent method based on event-triggered mechanism
• Fractional-order stochastic gradient descent method with momentum and energy for deep neural networks
• Computational performance of a generalized Descent Gradient Method based algorithm with conformable fractional-order derivatives
• An overview of the fractional-order gradient descent method and its applications

This page was built for publication: The gradient descent method from the perspective of fractional calculus

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5011114)