Stochastic gradient methods with preconditioned updates

DOI10.1007/S10957-023-02365-3MaRDI QIDQ6536836zbMATH OpenFDO

Authors Abdurakhmon Sadiev, Aleksandr Beznosikov, Abdulla Jasem Almansoori, Dmitry Kamzolov, Rachael Tappenden, M. Takáč

Publication date 14 May 2024

Published in Journal of Optimization Theory and Applications (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/S10957-023-02365-3

zbMATH Keywords

nonconvex optimizationoptimizationscaled methodsstochastic optimizationvariance reduction

Mathematics Subject Classification ID

Stochastic programming (90C15) Nonconvex programming, global optimization (90C26)

Recommendations

• Sub-sampled Newton methods
• SketchySGD: reliable stochastic optimization via randomized curvature estimates
• Preconditioning meets biased compression for efficient distributed optimization
• Minimizing finite sums with the stochastic average gradient
• Finite-sum smooth optimization with SARAH

Cites work

• A proximal stochastic gradient method with progressive variance reduction
• A Stochastic Approximation Method
• Adaptive subgradient methods for online learning and stochastic optimization
• An estimator for the diagonal of a matrix
• scientific article; zbMATH DE number 1243473 (Why is no real title available?)
• scientific article; zbMATH DE number 7255141 (Why is no real title available?)
• scientific article; zbMATH DE number 5060482 (Why is no real title available?)
• L-SVRG and L-Katyusha with arbitrary sampling
• Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
• Quasi-Newton Methods, Motivation and Theory
• Stochastic dual coordinate ascent methods for regularized loss minimization
• Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
• Understanding machine learning. From theory to algorithms

Cited in

(2)

• Preconditioning meets biased compression for efficient distributed optimization
• Local methods with adaptivity via scaling

This page was built for publication: Stochastic gradient methods with preconditioned updates

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6536836)