Accelerated Primal-Dual Algorithm for Distributed Non-convex Optimization

MaRDI QIDQ6375140zbMATH OpenFDO

Authors Shengjun Zhang, Colleen P. Bailey

Publication date 12 August 2021

Abstract: This paper investigates accelerating the convergence of distributed optimization algorithms on non-convex problems. We propose a distributed primal-dual stochastic gradient descent~(SGD) equipped with "powerball" method to accelerate. We show that the proposed algorithm achieves the linear speedup convergence rate $mathcalO(1/sqrtnT)$ for general smooth (possibly non-convex) cost functions. We demonstrate the efficiency of the algorithm through numerical experiments by training two-layer fully connected neural networks and convolutional neural networks on the MNIST dataset to compare with state-of-the-art distributed SGD algorithms and centralized SGD algorithms.

This page was built for publication: Accelerated Primal-Dual Algorithm for Distributed Non-convex Optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6375140)