On Nonconvex Decentralized Gradient Descent
From MaRDI portal
Publication:4622169
Abstract: Consensus optimization has received considerable attention in recent years. A number of decentralized algorithms have been proposed for {convex} consensus optimization. However, to the behaviors or consensus emph{nonconvex} optimization, our understanding is more limited. When we lose convexity, we cannot hope our algorithms always return global solutions though they sometimes still do sometimes. Somewhat surprisingly, the decentralized consensus algorithms, DGD and Prox-DGD, retain most other properties that are known in the convex setting. In particular, when diminishing (or constant) step sizes are used, we can prove convergence to a (or a neighborhood of) consensus stationary solution under some regular assumptions. It is worth noting that Prox-DGD can handle nonconvex nonsmooth functions if their proximal operators can be computed. Such functions include SCAD and quasi-norms, . Similarly, Prox-DGD can take the constraint to a nonconvex set with an easy projection. To establish these properties, we have to introduce a completely different line of analysis, as well as modify existing proofs that were used the convex setting.
Recommendations
- On the convergence of decentralized gradient descent
- On the Divergence of Decentralized Nonconvex Optimization
- A Fast Randomized Incremental Gradient Method for Decentralized Nonconvex Optimization
- On decentralized nonsmooth optimization
- Convergence results of a nested decentralized gradient method for non-strongly convex problems
- Decentralized Proximal Gradient Algorithms With Linear Convergence Rates
- Decentralized nonconvex optimization with guaranteed privacy and accuracy
- Decentralized Frank–Wolfe Algorithm for Convex and Nonconvex Problems
- Decentralized Learning With Lazy and Approximate Dual Gradients
- Decentralized Accelerated Gradient Methods With Increasing Penalty Parameters
Cited in
(28)- A fast proximal gradient algorithm for decentralized composite optimization over directed networks
- Convergence results of a nested decentralized gradient method for non-strongly convex problems
- Distributed stochastic nonsmooth nonconvex optimization
- A sigmoidal fractional derivative for regularization
- Moreau envelope augmented Lagrangian method for nonconvex optimization with linear constraints
- Optimal homotopy asymptotic method-least square for solving nonlinear fractional-order gradient-based dynamic system from an optimization problem
- Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization
- Distributed kernel ridge regression with communications
- Ghost penalties in nonconvex constrained optimization: diminishing stepsizes and iteration complexity
- scientific article; zbMATH DE number 7626753 (Why is no real title available?)
- On the Divergence of Decentralized Nonconvex Optimization
- Decentralized Accelerated Gradient Methods With Increasing Penalty Parameters
- An Improved Convergence Analysis for Decentralized Online Stochastic Non-Convex Optimization
- Second-order guarantees of distributed gradient algorithms
- A global dual error bound and its application to the analysis of linearly constrained nonconvex optimization
- Gradient-free federated learning methods with l₁ and l₂-randomization for non-smooth convex stochastic optimization problems
- Decentralized nonconvex optimization with guaranteed privacy and accuracy
- Federated learning for minimizing nonsmooth convex loss functions
- Decentralized learning over a network with Nyström approximation using SGD
- A decentralized smoothing quadratic regularization algorithm for composite consensus optimization with non-Lipschitz singularities
- DIMIX: Diminishing Mixing for Sloppy Agents
- A variance-reduced stochastic gradient tracking algorithm for decentralized optimization with orthogonality constraints
- Last-iterate convergence of shuffling momentum gradient method under the Kurdyka-Lojasiewicz inequality
- A decentralized proximal gradient tracking algorithm for composite optimization on Riemannian manifolds
- Decentralized projected Riemannian gradient method for smooth optimization on compact submanifolds embedded in the Euclidean space
- Differentially private and communication-efficient distributed nonconvex optimization algorithms
- A penalty-based method for communication-efficient decentralized bilevel programming
- Clipped stochastic gradient tracking for locally smooth functions
This page was built for publication: On Nonconvex Decentralized Gradient Descent
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4622169)
