S-NEAR-DGD: A Flexible Distributed Stochastic Gradient Method for Inexact Communication

DOI10.1109/TAC.2022.3151734MaRDI QIDQ6093049zbMATH OpenOpenAlexFDO

Authors Ermin Wei

Publication date 6 September 2023

Published in IEEE Transactions on Automatic Control (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2102.00121

zbMATH Keywords

distributed optimizationnetwork optimizationquantizationstochastic optimization

Mathematics Subject Classification ID

Systems theory; control (93-XX)

Abstract: We present and analyze a stochastic distributed method (S-NEAR-DGD) that can tolerate inexact computation and inaccurate information exchange to alleviate the problems of costly gradient evaluations and bandwidth-limited communication in large-scale systems. Our method is based on a class of flexible, distributed first order algorithms that allow for the trade-off of computation and communication to best accommodate the application setting. We assume that all the information exchange between nodes is subject to random distortion and that only stochastic approximations of the true gradients are available. Our theoretical results prove that the proposed algorithm converges linearly in expectation to a neighborhood of the optimal solution for strongly convex objective functions with Lipschitz gradients. We characterize the dependence of this neighborhood on algorithm and network parameters, the quality of the communication channel and the precision of the stochastic gradient approximations used. Finally, we provide numerical results to evaluate the empirical performance of our method.

Recommendations

• Communication-Adaptive Stochastic Gradient Methods for Distributed Learning
• A stochastic averaging gradient algorithm with multi‐step communication for distributed optimization
• scientific article; zbMATH DE number 7306853
• A Continuous-Time Analysis of Distributed Stochastic Gradient
• scientific article; zbMATH DE number 7307473
• Convergence of Asynchronous Distributed Gradient Methods Over Stochastic Networks
• D-ADMM: A Communication-Efficient Distributed Algorithm for Separable Optimization
• A distributed flexible delay-tolerant proximal gradient algorithm
• Improving the convergence of distributed gradient descent via inexact average consensus
• Convergence Rates of Distributed Gradient Methods Under Random Quantization: A Stochastic Approximation Approach

Cited in

(2)

• Efficient gradient tracking algorithms for distributed optimization problems with inexact communication
• Privacy-preserving gradient tracking via state decomposition for directed multi-agent optimization

This page was built for publication: S-NEAR-DGD: A Flexible Distributed Stochastic Gradient Method for Inexact Communication

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6093049)