Decentralized Asynchronous Non-convex Stochastic Optimization on Directed Graphs

arXiv2110.10406MaRDI QIDQ6380833FDOQ6380833

Authors: Vyacheslav Kungurtsev, Mahdi Morafah, Tara Javidi, Gesualdo Scutari 

Publication date: 20 October 2021

Abstract: Distributed Optimization is an increasingly important subject area with the rise of multi-agent control and optimization. We consider a decentralized stochastic optimization problem where the agents on a graph aim to asynchronously optimize a collective (additive) objective function consisting of agents' individual (possibly non-convex) local objective functions. Each agent only has access to a noisy estimate of the gradient of its own function (one component of the sum of objective functions). We proposed an asynchronous distributed algorithm for such a class of problems. The algorithm combines stochastic gradients with tracking in an asynchronous push-sum framework and obtain the standard sublinear convergence rate for general non-convex functions, matching the rate of centralized stochastic gradient descent SGD. Our experiments on a non-convex image classification task using convolutional neural network validate the convergence of our proposed algorithm across different number of nodes and graph connectivity percentages.

This page was built for publication: Decentralized Asynchronous Non-convex Stochastic Optimization on Directed Graphs

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6380833)