Exact Diffusion for Distributed Optimization and Learning—Part I: Algorithm Development
From MaRDI portal
Abstract: This work develops a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown in Part II to have a wider stability range and superior convergence performance than the EXTRA strategy. The exact diffusion solution is applicable to non-symmetric left-stochastic combination matrices, while many earlier developments on exact consensus implementations are limited to doubly-stochastic matrices; these latter matrices impose stringent constraints on the network topology. The derivation of the exact diffusion strategy in this work relies on reformulating the aggregate optimization problem as a penalized problem and resorting to a diagonally-weighted incremental construction. Detailed stability and convergence analyses are pursued in Part II and are facilitated by examining the evolution of the error dynamics in a transformed domain. Numerical simulations illustrate the theoretical conclusions.
Recommendations
- Exact Diffusion for Distributed Optimization and Learning—Part II: Convergence Analysis
- Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks
- Gradient-free distributed optimization with exact convergence
- scientific article; zbMATH DE number 7306853
- Distributed Stochastic Optimization via Matrix Exponential Learning
- Exact spectral-like gradient method for distributed optimization
- Random gradient extrapolation for distributed and stochastic optimization
- On the convergence of exact distributed generalisation and acceleration algorithm for convex optimisation
- Distributed asynchronous deterministic and stochastic gradient optimization algorithms
- Distributed optimization and statistical learning via the alternating direction method of multipliers
Cited in
(16)- On the linear convergence of two decentralized algorithms
- Linear convergence of primal-dual gradient methods and their performance in distributed optimization
- Correction-based diffusion LMS algorithms for distributed estimation
- Proximal nested primal-dual gradient algorithms for distributed constraint-coupled composite optimization
- Distributed Stochastic Optimization via Matrix Exponential Learning
- On the convergence of exact distributed generalisation and acceleration algorithm for convex optimisation
- scientific article; zbMATH DE number 7307473 (Why is no real title available?)
- A machine-learning-accelerated distributed LBFGS method for field development optimization: algorithm, validation, and applications
- Linear convergence rate analysis of a class of exact first-order distributed methods for weight-balanced time-varying networks and uncoordinated step sizes
- Golden ratio proximal gradient ADMM for distributed composite convex optimization
- Optimal gradient tracking for decentralized optimization
- Distributed inexact Newton method with adaptive step sizes
- On graphs with finite-time consensus and their use in gradient tracking
- Convergence of an accelerated distributed optimisation algorithm over time-varying directed networks
- Local adapt-then-combine algorithms for distributed nonsmooth optimization: achieving provable communication acceleration
- Decentralized sparse linear regression via gradient-tracking
This page was built for publication: Exact Diffusion for Distributed Optimization and Learning—Part I: Algorithm Development
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4628229)
