Stochastic modified flows for Riemannian stochastic gradient descent

DOI10.1137/24M163863XMaRDI QIDQ6658239zbMATH OpenFDO

Authors Benjamn Gess, Sebastian Kassing, Nimit Rana

Publication date 8 January 2025

Published in SIAM Journal on Control and Optimization (Search for Journal in Brave)

zbMATH Keywords

supervised learningdiffusion approximationweak errorRiemannian gradient flowRiemannian stochastic gradient descent

Mathematics Subject Classification ID

Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Numerical mathematical programming methods (65K05) Stochastic approximation (62L20) Diffusion processes and stochastic analysis on manifolds (58J65)

Cites work

• scientific article; zbMATH DE number 3812549 (Why is no real title available?)
• scientific article; zbMATH DE number 43057 (Why is no real title available?)
• scientific article; zbMATH DE number 5223994 (Why is no real title available?)
• A Systematic Approach to Lyapunov Analyses of Continuous-Time Models in Convex Optimization
• An Introduction to Optimization on Smooth Manifolds
• Convergence rates for the stochastic gradient descent method for non-convex objective functions
• Cooling down stochastic differential equations: Almost sure convergence
• Cylindrical Wiener processes
• Existence, uniqueness and regularity of the projection onto differentiable manifolds
• Fisher information distance: a geometrical reading
• Gradient algorithms for principal component analysis
• Lie groups
• Normally hyperbolic invariant manifolds. The noncompact case
• On closed-form expressions for the Fisher-Rao distance
• On minimal representations of shallow ReLU networks
• On the diffusion approximation of nonconvex stochastic gradient descent
• On uniform-in-time diffusion approximation for stochastic gradient descent
• Projection-like retractions on matrix manifolds
• Properties at infinity of diffusion semigroups and stochastic flows via weak uniform covers
• Quadratic optimization with orthogonality constraint: explicit Łojasiewicz exponent and linear convergence of retraction-based line-search and stochastic variance-reduced gradient methods
• Robust implicit regularization via weight normalization
• Semigroups of stochastic gradient descent and online principal component analysis: properties and diffusion approximations
• Stochastic Equations in Infinite Dimensions
• Stochastic Gradient Descent on Riemannian Manifolds
• Stochastic analysis on manifolds
• Stochastic approximation on Riemannian manifolds
• Stochastic gradient descent with noise of machine learning type. II: Continuous time analysis
• Stochastic modified equations and dynamics of stochastic gradient algorithms. I: Mathematical foundations
• Stochastic partial differential equations: an introduction
• Strong \(p\)-completeness of stochastic differential equations and the existence of smooth flows on noncompact manifolds
• The Geometry of Algorithms with Orthogonality Constraints
• The central limit problem for geodesic random walks
• Uniform-in-time weak error analysis for stochastic gradient descent algorithms via diffusion approximation

This page was built for publication: Stochastic modified flows for Riemannian stochastic gradient descent

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6658239)