Variance-reduced reshuffling gradient descent for nonconvex optimization: centralized and distributed algorithms

DOI10.1016/J.AUTOMATICA.2024.111954MaRDI QIDQ6659241zbMATH OpenFDO

Authors Xia Jiang, Xianlin Zeng, Lihua Xie, Jian Sun, Jie Chen

Publication date 8 January 2025

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/J.AUTOMATICA.2024.111954

zbMATH Keywords

multi-agent networksnonconvex optimizationreshuffling gradient descentsampling without replacementvariance reduction

Mathematics Subject Classification ID

Systems theory; control (93-XX)

Recommendations

• Stochastic alternating structure-adapted proximal gradient descent method with variance reduction for nonconvex nonsmooth optimization
• Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization
• Stochastic nested variance reduction for nonconvex optimization
• An Accelerated Stochastic ADMM for Nonconvex and Nonsmooth Finite-Sum Optimization
• Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization

Cites work

• A unified distributed method for constrained networked optimization via saddle-point dynamics
• Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
• Convergence of Random Reshuffling under the Kurdyka–Łojasiewicz Inequality
• Decentralized nonconvex optimization with guaranteed privacy and accuracy
• Distributed random reshuffling over networks
• Distributed Stochastic Consensus Optimization With Momentum for Nonconvex Nonsmooth Problems
• Distributed stochastic subgradient projection algorithms for convex optimization
• First-order methods almost always avoid strict saddle points
• scientific article; zbMATH DE number 7626722 (Why is no real title available?)
• Minimizing finite sums with the stochastic average gradient
• Multicomposite nonconvex optimization for training deep neural networks
• Nonlinear programming
• Proximal ADMM for nonconvex and nonsmooth optimization
• Proximal alternating linearized minimization for nonconvex and nonsmooth problems
• Variance reduction in covariance based realization algorithm with application to closed-loop data
• Variance-Reduced Stochastic Learning Under Random Reshuffling
• Variance-reduced stochastic quasi-Newton methods for decentralized learning
• Why random reshuffling beats stochastic gradient descent
• Zeroth-order algorithms for stochastic distributed nonconvex optimization

This page was built for publication: Variance-reduced reshuffling gradient descent for nonconvex optimization: centralized and distributed algorithms

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6659241)