On the diffusion approximation of nonconvex stochastic gradient descent

DOI10.4310/AMSA.2019.V4.N1.A1zbMath1421.90121arXiv1705.07562WikidataQ128319363 ScholiaQ128319363MaRDI QIDQ1734292

Wenqing Hu, Lei Li, Chris Junchi Li, Jian-Guo Liu

Publication date: 27 March 2019

Published in: Annals of Mathematical Sciences and Applications (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1705.07562

zbMATH Keywords

diffusion approximationnonconvex optimizationstationary pointsstochastic gradient descentbatch size

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26)

Related Items (12)

Approximation to stochastic variance reduced gradient Langevin dynamics by stochastic delay differential equations ⋮ The effective noise of stochastic gradient descent ⋮ A convergence analysis of the perturbed compositional gradient flow: averaging principle and normal deviations ⋮ A probability approximation framework: Markov process approach ⋮ A Random-Batch Monte Carlo Method for Many-Body Systems with Singular Kernels ⋮ A Continuous-Time Analysis of Distributed Stochastic Gradient ⋮ Random batch methods (RBM) for interacting particle systems ⋮ Stochastic differential equation approximations of generative adversarial network training and its long-run behavior ⋮ Analysis of stochastic gradient descent in continuous time ⋮ On the fast convergence of random perturbations of the gradient flow ⋮ Unnamed Item ⋮ Numerical analysis for inchworm Monte Carlo method: Sign problem and error growth

This page was built for publication: On the diffusion approximation of nonconvex stochastic gradient descent