Pipelined model parallelism: complexity results and memory considerations

DOI10.1007/978-3-030-85665-6_12MaRDI QIDQ6487190zbMATH OpenFDO

Authors Olivier Beaumont, Lionel Eyraud-Dubois, Alena Shilova

Publication date 31 March 2022

Full work available at URL https://doi.org/10.1007/978-3-030-85665-6_12

Mathematics Subject Classification ID

Performance evaluation, queueing, and scheduling in the context of computer systems (68M20) Analysis of algorithms and problem complexity (68Q25) Artificial neural networks and deep learning (68T07) Parallel algorithms in computer science (68W10)

Recommendations

• Optimal memory-aware backpropagation of deep join networks
• Using machine learning to model the training scalability of convolutional neural networks on clusters of GPUs
• Efficient and systematic partitioning of large and deep neural networks for parallelization
• Distributed Deep Learning on Heterogeneous Computing Resources Using Gossip Communication
• Computing with pipelined block transfer

Cites work

• scientific article; zbMATH DE number 3639144 (Why is no real title available?)
• Tight results for next fit and worst fit with resource augmentation

Cited in

(4)

• Pipelined iterative methods for shared memory machines
• Optimal memory-aware backpropagation of deep join networks
• Using machine learning to model the training scalability of convolutional neural networks on clusters of GPUs
• Efficient and systematic partitioning of large and deep neural networks for parallelization

This page was built for publication: Pipelined model parallelism: complexity results and memory considerations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6487190)