High-Performance Tensor Contraction without Transposition

DOI10.1137/16M108968XMaRDI QIDQ4600011zbMATH OpenOpenAlexFDO

Authors Devin A. Matthews

Publication date 5 January 2018

Published in SIAM Journal on Scientific Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1607.00291

zbMATH Keywords

performancemultilinear algebramatrix multiplicationhigh-performance computingtensor contractiontensor-to-matrix transformation

Mathematics Subject Classification ID

Complexity and performance of numerical algorithms (65Y20) Multilinear algebra, tensor calculus (15A69)

Abstract: Tensor computations--in particular tensor contraction (TC)--are important kernels in many scientific computing applications. Due to the fundamental similarity of TC to matrix multiplication (MM) and to the availability of optimized implementations such as the BLAS, tensor operations have traditionally been implemented in terms of BLAS operations, incurring both a performance and a storage overhead. Instead, we implement TC using the flexible BLIS framework, which allows for transposition (reshaping) of the tensor to be fused with internal partitioning and packing operations, requiring no explicit transposition operations or additional workspace. This implementation, TBLIS, achieves performance approaching that of MM, and in some cases considerably higher than that of traditional TC. Our implementation supports multithreading using an approach identical to that used for MM in BLIS, with similar performance characteristics. The complexity of managing tensor-to-matrix transformations is also handled automatically in our approach, greatly simplifying its use in scientific applications.

Recommendations

• High performance rearrangement and multiplication routines for sparse tensor arithmetic
• Tensorisation of vectors and their efficient convolution
• Efficient Tensor Decompositions
• Faster tensor canonicalization
• Languages and Compilers for Parallel Computing
• Exploiting symmetry in tensors for high performance: multiplication with symmetric tensors
• TTC: a high-performance compiler for tensor transpositions
• Fast bilinear algorithms for symmetric tensor contractions
• Strassen's Algorithm for Tensor Contraction

Cites work

• scientific article; zbMATH DE number 2045481 (Why is no real title available?)
• A set of level 3 basic linear algebra subprograms
• Algorithm 862
• An extended set of FORTRAN basic linear algebra subprograms
• Anatomy of high-performance matrix multiplication
• Applied Multiway Data Analysis
• BLIS: a framework for rapidly instantiating BLAS functionality
• Basic Linear Algebra Subprograms for Fortran Usage
• Design of a high-performance GEMM-like tensor-tensor multiplication
• TTC: a high-performance compiler for tensor transpositions
• Tensor Decompositions and Applications
• Towards an efficient use of the BLAS library for multilinear tensor contractions

Cited in

(9)

• Towards an efficient use of the BLAS library for multilinear tensor contractions
• Computational Science – ICCS 2005
• Languages and Compilers for Parallel Computing
• Design of a high-performance GEMM-like tensor-tensor multiplication
• Strassen's Algorithm for Tensor Contraction
• High performance rearrangement and multiplication routines for sparse tensor arithmetic
• Spin summations: a high-performance perspective
• TTC: a high-performance compiler for tensor transpositions
• A \(\mu\)-mode BLAS approach for multidimensional tensor-structured problems

Describes a project that uses

Uses Software

• AUGEM
• TTC
• BLIS
• Algorithm 862
• NumPy
• Eigen
• Blitz++ library
• BLAS
• Blitz++

This page was built for publication: High-Performance Tensor Contraction without Transposition

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4600011)