Scaling up stochastic gradient descent for non-convex optimisation

DOI10.1007/S10994-022-06243-3MaRDI QIDQ6097095zbMATH OpenOpenAlexFDO

Authors Saad Mohamad, Hamad Alamri, Abdelhamid Bouchachia

Publication date 12 June 2023

Published in Machine Learning (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2210.02882

zbMATH Keywords

deep reinforcement learningdistributed and parallel computationlarge scale non-convex optimisationstochastic gradient descentvariational inference

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Stochastic gradient descent (SGD) is a widely adopted iterative method for optimizing differentiable objective functions. In this paper, we propose and discuss a novel approach to scale up SGD in applications involving non-convex functions and large datasets. We address the bottleneck problem arising when using both shared and distributed memory. Typically, the former is bounded by limited computation resources and bandwidth whereas the latter suffers from communication overheads. We propose a unified distributed and parallel implementation of SGD (named DPSGD) that relies on both asynchronous distribution and lock-free parallelism. By combining two strategies into a unified framework, DPSGD is able to strike a better trade-off between local computation and communication. The convergence properties of DPSGD are studied for non-convex problems such as those arising in statistical modelling and machine learning. Our theoretical analysis shows that DPSGD leads to speed-up with respect to the number of cores and number of workers while guaranteeing an asymptotic convergence rate of $O(1/sqrtT)$ given that the number of cores is bounded by $T^{1/4}$ and the number of workers is bounded by $T^{1/2}$ where $T$ is the number of iterations. The potential gains that can be achieved by DPSGD are demonstrated empirically on a stochastic variational inference problem (Latent Dirichlet Allocation) and on a deep reinforcement learning (DRL) problem (advantage actor critic - A2C) resulting in two algorithms: DPSVI and HSA2C. Empirical results validate our theoretical findings. Comparative studies are conducted to show the performance of the proposed DPSGD against the state-of-the-art DRL algorithms.

Cites work

• 10.1162/jmlr.2003.3.4-5.993
• A Stochastic Approximation Method
• An introduction to variational methods for graphical models
• Distributed asynchronous deterministic and stochastic gradient optimization algorithms
• scientific article; zbMATH DE number 6377992 (Why is no real title available?)
• Large-scale machine learning with stochastic gradient descent
• Optimal distributed online prediction using mini-batches
• Optimization methods for large-scale machine learning
• Reinforcement learning. An introduction
• Robust Stochastic Approximation Approach to Stochastic Programming
• Simple statistical gradient-following algorithms for connectionist reinforcement learning
• Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
• The vectorization of ITPACK 2C

Cited in

(1)

• Convergence of Adam for non-convex objectives: relaxed hyperparameters and non-ergodic case

This page was built for publication: Scaling up stochastic gradient descent for non-convex optimisation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6097095)