Decentralized consensus algorithm with delayed and stochastic gradients
From MaRDI portal
Abstract: We analyze the convergence of decentralized consensus algorithm with delayed gradient information across the network. The nodes in the network privately hold parts of the objective function and collaboratively solve for the consensus optimal solution of the total objective while they can only communicate with their immediate neighbors. In real-world networks, it is often difficult and sometimes impossible to synchronize the nodes, and therefore they have to use stale gradient information during computations. We show that, as long as the random delays are bounded in expectation and a proper diminishing step size policy is employed, the iterates generated by decentralized gradient descent method converge to a consensual optimal solution. Convergence rates of both objective and consensus are derived. Numerical results on a number of synthetic problems and real-world seismic tomography datasets in decentralized sensor networks are presented to show the performance of the method.
Recommendations
- On the convergence of decentralized gradient descent
- A randomized incremental primal-dual method for decentralized consensus optimization
- DSA: decentralized double stochastic averaging gradient algorithm
- EXTRA: an exact first-order algorithm for decentralized consensus optimization
- Distributed asynchronous deterministic and stochastic gradient optimization algorithms
Cites work
- Adaptation, learning, and optimization over networks
- An Asynchronous Parallel Stochastic Coordinate Descent Algorithm
- Asynchronous Distributed ADMM for Large-Scale Optimization—Part I: Algorithm and<?Pub _newline ?>Convergence Analysis
- Asynchronous Distributed ADMM for Large-Scale Optimization—Part II: Linear Convergence Analysis and Numerical Performance
- Broadcast Gossip Algorithms for Consensus
- Consensus of Multi-Agent Systems With Diverse Input and Communication Delays
- Consensus Problems in Networks of Agents With Switching Topology and Time-Delays
- Consensus-based distributed support vector machines
- Convergence Rate of Distributed ADMM Over Networks
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Distributed Optimization Over Time-Varying Directed Graphs
- Distributed Subgradient Methods for Multi-Agent Optimization
- DSA: decentralized double stochastic averaging gradient algorithm
- Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
- Explicit Convergence Rate of a Distributed Alternating Direction Method of Multipliers
- EXTRA: an exact first-order algorithm for decentralized consensus optimization
- Fast Distributed Gradient Methods
- Fast linear iterations for distributed averaging
- Linear Convergence Rate of a Class of Distributed Augmented Lagrangian Algorithms
- Multi-Agent Distributed Optimization via Inexact Consensus ADMM
- On the convergence of decentralized gradient descent
- On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
- Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs
Cited in
(11)- Optimal tradeoff between instantaneous and delayed neighbor information in consensus algorithms
- Distributed stochastic gradient tracking methods
- DSA: decentralized double stochastic averaging gradient algorithm
- On the convergence of decentralized gradient descent
- Robust asynchronous stochastic gradient-push: asymptotically optimal and network-independent performance for strongly convex functions
- A randomized incremental primal-dual method for decentralized consensus optimization
- Distributed adaptive consensus protocol with decaying gains
- A code-based distributed gradient descent scheme for decentralized convex optimization
- Non-ergodic linear convergence property of the delayed gradient descent under the strongly convexity and the Polyak-Łojasiewicz condition
- Federated dual averaging learning algorithm with delayed gradients for composite optimization
- Privacy-preserving gradient tracking via state decomposition for directed multi-agent optimization
This page was built for publication: Decentralized consensus algorithm with delayed and stochastic gradients
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4641647)
