A unified momentum-based paradigm of decentralized SGD for non-convex models and heterogeneous data

DOI10.1016/J.ARTINT.2024.104130MaRDI QIDQ6579292zbMATH OpenFDO

Authors H. Du, Chaoqian Cheng, Chengdong Ni

Publication date 25 July 2024

Published in Artificial Intelligence (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/J.ARTINT.2024.104130

zbMATH Keywords

decentralized traininggradient trackingheterogeneous datamomentum techniquenonconvex optimization

Mathematics Subject Classification ID

Computational aspects of data analysis and big data (68T09) Nonconvex programming, global optimization (90C26)

Recommendations

• Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization
• DESTRESS: Computation-Optimal and Communication-Efficient Decentralized Nonconvex Finite-Sum Optimization
• Decentralized nonconvex optimization with guaranteed privacy and accuracy
• A decentralized Nesterov gradient method for stochastic optimization over unbalanced directed networks
• An Optimal Algorithm for Decentralized Finite-Sum Optimization

Cites work

• An Improved Convergence Analysis for Decentralized Online Stochastic Non-Convex Optimization
• Distributed stochastic gradient tracking methods
• Some methods of speeding up the convergence of iteration methods

Cited in

(1)

• A unified theoretical framework for the last-iterate convergence of stochastic adaptive optimization

This page was built for publication: A unified momentum-based paradigm of decentralized SGD for non-convex models and heterogeneous data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6579292)