Distributed Deep Learning on Heterogeneous Computing Resources Using Gossip Communication

DOI10.1007/978-3-030-41032-2_25MaRDI QIDQ3297711zbMATH OpenOpenAlexFDO

Authors Dobromir Georgiev, T. Gurov

Publication date 20 July 2020

Published in Large-Scale Scientific Computing (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/978-3-030-41032-2_25

zbMATH Keywords

deep learninggossip communicationheterogeneous high-performance computing

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Network design and communication in computer systems (68M10) Distributed systems (68M14)

Recommendations

• GADMM: fast and communication efficient framework for distributed machine learning
• scientific article; zbMATH DE number 1928603
• Distributed Machine Learning and Gradient Optimization
• Distributed Newton Methods for Deep Neural Networks
• Communication-Adaptive Stochastic Gradient Methods for Distributed Learning
• Distributed Reinforcement Learning via Gossip
• Efficient protocols for distributed classification and optimization
• Deep distributed convolutional neural networks: universality
• scientific article; zbMATH DE number 1011550
• Distributed kernel-based gradient descent algorithms

Cited in

(3)

• Using machine learning to model the training scalability of convolutional neural networks on clusters of GPUs
• Efficient and systematic partitioning of large and deep neural networks for parallelization
• Pipelined model parallelism: complexity results and memory considerations

Describes a project that uses

Uses Software

• CIFAR

This page was built for publication: Distributed Deep Learning on Heterogeneous Computing Resources Using Gossip Communication

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3297711)