PyTorch: An Imperative Style, High-Performance Deep Learning Library

DOI10.48550/ARXIV.1912.01703MaRDI QIDQ77737arXivFDO

Authors Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach Devito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, Soumith Chintala

Publication date 3 December 2019

Published in arXiv

Full work available at URL https://doi.org/10.48550/ARXIV.1912.01703

arXiv classification

cs.LGcs.MSstat.ML

Abstract: Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.

Cited in

(11)

• torch
• A Sub-linear Low-Rank Solver for Poisson's Equation using Machine Learning Frameworks for GPU Acceleration
• JAX-FVM: A differentiable, entropy-stable finite volume solver on unstructured meshes for compressible flows
• Design Choices in Splitting-Based Self-Supervised Sparse-View CT Reconstruction
• Multiscale Mixed-Dimensional Simulation via Domain Decomposition and Non-Intrusive Neural Model Order Reduction
• Lightweight return-mapping surrogates for multiscale plasticity: a practical guide
• Automated Numerical Stability Analysis of Deep Learning Operators
• It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
• Data-Driven Model Order Reduction with pyMOR
• Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond
• Batched Wavefront Sweeps for Linear Transport Uncertainty Quantification

This page was built for publication: PyTorch: An Imperative Style, High-Performance Deep Learning Library

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q77737)