Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation
From MaRDI portal
Publication:6420105
arXiv2212.05159MaRDI QIDQ6420105FDOQ6420105
Authors: Nicolas Nytko, Ali Taghibakhshi, Tareq Uz Zaman, Scott MacLachlan, Luke N. Olson, Matt West 
Publication date: 9 December 2022
Abstract: Sparse matrix representations are ubiquitous in computational science and machine learning, leading to significant reductions in compute time, in comparison to dense representation, for problems that have local connectivity. The adoption of sparse representation in leading ML frameworks such as PyTorch is incomplete, however, with support for both automatic differentiation and GPU acceleration missing. In this work, we present an implementation of a CSR-based sparse matrix wrapper for PyTorch with CUDA acceleration for basic matrix operations, as well as automatic differentiability. We also present several applications of the resulting sparse kernels to optimization problems, demonstrating ease of implementation and performance measurements versus their dense counterparts.
Has companion code repository: https://github.com/nicknytko/numml
This page was built for publication: Optimized Sparse Matrix Operations for Reverse Mode Automatic Differentiation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6420105)
