Optimal memory-aware backpropagation of deep join networks

DOI10.1098/RSTA.2019.0049MaRDI QIDQ4993499zbMATH OpenOpenAlexWikidataFDO

Authors Olivier Beaumont, Julien Herrmann, Guillaume Pallez, Alena Shilova

Publication date 15 June 2021

Published in Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences (Search for Journal in Brave)

Full work available at URL https://hal.inria.fr/hal-02401105/file/rsta_rev2.pdf

zbMATH Keywords

artificial intelligencebackpropagationdeep learningmemorypebble gametheory of computing

Mathematics Subject Classification ID

Graph theory (including graph drawing) in computer science (68R10) Artificial neural networks and deep learning (68T07)

Recommendations

• Pipelined model parallelism: complexity results and memory considerations
• scientific article; zbMATH DE number 166796
• The Data-Flow Equations of Checkpointing in Reverse Automatic Differentiation
• Convergence of back-propagation with momentum for network architectures with skip connections
• Using machine learning to model the training scalability of convolutional neural networks on clusters of GPUs

Cites work

• Algorithm 799: revolve
• An Application of Generalized Tree Pebbling to Sparse Matrix Factorization
• Complete Register Allocation Problems
• Deep metric learning using triplet network
• H-revolve: a framework for adjoint computation on synchronous hierarchical platforms
• scientific article; zbMATH DE number 4141383 (Why is no real title available?)
• Optimal memory-aware backpropagation of deep join networks
• Optimal multistage algorithm for adjoint computation
• Scheduling series-parallel task graphs to minimize peak memory

Cited in

(3)

• Optimal memory-aware backpropagation of deep join networks
• Saving memory space in deep neural networks by recomputing: a survey
• Pipelined model parallelism: complexity results and memory considerations

This page was built for publication: Optimal memory-aware backpropagation of deep join networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4993499)