DESTRESS: Computation-Optimal and Communication-Efficient Decentralized Nonconvex Finite-Sum Optimization

DOI10.1137/21M1450677MaRDI QIDQ5095229zbMATH OpenOpenAlexFDO

Authors Boyue Li, Zhize Li, Yuejie Chi

Publication date 5 August 2022

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2110.01165

zbMATH Keywords

decentralized optimizationnonconvex finite-sum optimizationstochastic recursive gradient methods

Mathematics Subject Classification ID

Communication complexity, information complexity (68Q11) Analysis of algorithms and problem complexity (68Q25) Computational aspects of data analysis and big data (68T09)

Recommendations

• Communication-efficient algorithms for decentralized and stochastic optimization
• An Optimal Algorithm for Decentralized Finite-Sum Optimization
• Decentralized online convex optimization with compressed communications
• scientific article; zbMATH DE number 7307473
• Decentralized ADMM with compressed and event-triggered communication

Cites work

• A Fast Randomized Incremental Gradient Method for Decentralized Nonconvex Optimization
• Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs
• Balancing Communication and Computation in Distributed Optimization
• Chebyshev acceleration of iterative refinement
• Convergence of Distributed Stochastic Variance Reduced Methods Without Sampling Extra Data
• Decentralized Accelerated Gradient Methods With Increasing Penalty Parameters
• Discrete-time dynamic average consensus
• Distributed Control of Multiconsensus
• Distributed stochastic variance reduced gradient methods by sampling extra data with replacement
• Distributed Subgradient Methods for Multi-Agent Optimization
• EXTRA: an exact first-order algorithm for decentralized consensus optimization
• Fast decentralized nonconvex finite-sum optimization with recursive variance reduction
• Fast linear iterations for distributed averaging
• Finite-sum smooth optimization with SARAH
• Harnessing Smoothness to Accelerate Distributed Optimization
• scientific article; zbMATH DE number 7307473 (Why is no real title available?)
• On the Convergence of Nested Decentralized Gradient Methods With Multiple Consensus and Gradient Steps

Cited in

(5)

• An Optimal Algorithm for Decentralized Finite-Sum Optimization
• Decentralized personalized federated learning: lower bounds and optimal algorithm for all personalization modes
• A unified momentum-based paradigm of decentralized SGD for non-convex models and heterogeneous data
• Minibatch and local SGD: algorithmic stability and linear speedup in generalization
• Escaping saddle points in distributed nonconvex optimization via cubic regularization

Describes a project that uses

Uses Software

• SpiderBoost
• Saga
• MNIST

This page was built for publication: DESTRESS: Computation-Optimal and Communication-Efficient Decentralized Nonconvex Finite-Sum Optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5095229)