Stochastic modified equations for the asynchronous stochastic gradient descent

DOI10.1093/IMAIAI/IAZ030MaRDI QIDQ5006542zbMATH OpenOpenAlexWikidataFDO

Authors Jing An, Lexing Ying, Jianfeng Lu

Publication date 16 August 2021

Published in Information and Inference: A Journal of the IMA (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1805.08244

zbMATH Keywords

asynchronous stochastic gradient descentoptimal controlstochastic modified equations

Mathematics Subject Classification ID

Optimality conditions for problems involving randomness (49K45) Numerical solutions to stochastic differential and integral equations (65C30)

Recommendations

• Stochastic modified equations and dynamics of stochastic gradient algorithms. I: Mathematical foundations
• A sharp convergence rate for a model equation of the asynchronous stochastic gradient descent
• Parallel and distributed asynchronous adaptive stochastic gradient methods
• Analysis of stochastic gradient descent in continuous time
• Asynchronous SGD with stale gradient dynamic adjustment for deep learning training

Cited in

(5)

• An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization
• Stochastic modified equations and dynamics of stochastic gradient algorithms. I: Mathematical foundations
• Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach
• Applications of Fokker Planck equations in machine learning algorithms
• On large batch training and sharp minima: a Fokker-Planck perspective

This page was built for publication: Stochastic modified equations for the asynchronous stochastic gradient descent

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5006542)