PyTorch: An Imperative Style, High-Performance Deep Learning Library

DOI10.48550/ARXIV.1912.01703MaRDI QIDQ77737arXivFDO

Authors Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach Devito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, Soumith Chintala

Publication date 3 December 2019

Published in arXiv

Full work available at URL https://doi.org/10.48550/ARXIV.1912.01703

arXiv classification

cs.LGcs.MSstat.ML

Abstract: Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.

Cited in

(23)

• A Sub-linear Low-Rank Solver for Poisson's Equation using Machine Learning Frameworks for GPU Acceleration
• JAX-FVM: A differentiable, entropy-stable finite volume solver on unstructured meshes for compressible flows
• Design Choices in Splitting-Based Self-Supervised Sparse-View CT Reconstruction
• Multiscale Mixed-Dimensional Simulation via Domain Decomposition and Non-Intrusive Neural Model Order Reduction
• Lightweight return-mapping surrogates for multiscale plasticity: a practical guide
• Automated Numerical Stability Analysis of Deep Learning Operators
• It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
• Data-Driven Model Order Reduction with pyMOR
• Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond
• Rapid Earthquake-to-Tsunami Waveform Generation via Large-Scale Multi-GPU FFT Convolution Applied to the Cascadia Subduction Zone
• Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks
• QGPINNs: A Physics-Informed Neural Network Framework for Nonlocal Differential Equations on Quantum Graphs
• GRADSOLVE: fast exact gradients for ODE ensembles on GPUs
• Vectorizer: Vectorizing NumPy Programs with Shape-Guided Rewrite
• Efficient computation of the asymptotics of extensive-rank HCIZ integrals
• A Systematic Analysis of Automatic Differentiation versus Discretization-based Constraints for Physics-Informed PDE Solvers
• Batched Wavefront Sweeps for Linear Transport Uncertainty Quantification
• Well-posed by Design: Learning Constitutive Laws from Velocity Data using Convex Neural Network Potentials
• Learning a generalized Navier-Stokes model beyond the continuum regime
• Residual-Based Adaptive Refinement in Hybrid Variational-Collocation PINNs for Singular Elliptic PDEs with Mixed Dirichlet-Neumann Boundary Conditions
• LowNet: Looped Wavelet Neural Networks with 24 Learnable Parameters for Solving Poisson Equations
• Structured Pruning of Feedback Delay Networks
• Sensitivity-Guided Parameter Estimation of Bubbly Liquid via a Nonlinear Wave Equation and Physics-InformedNeuralNetworks

This page was built for publication: PyTorch: An Imperative Style, High-Performance Deep Learning Library

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q77737)