Design of a high-performance GEMM-like tensor-tensor multiplication

DOI10.1145/3157733MaRDI QIDQ4581393zbMATH OpenOpenAlexWikidataFDO

Authors Paul Springer, Paolo Bientinesi

Publication date 17 August 2018

Published in ACM Transactions on Mathematical Software (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1607.00145

zbMATH Keywords

domain-specific code generatorhigh-performance computingmatrix-matrix multiplicationtensor contractions

Mathematics Subject Classification ID

Vector and tensor algebra, theory of invariants (15A72) Numerical linear algebra (65F99) Parallel numerical computation (65Y05)

Abstract: We present "GEMM-like Tensor-Tensor multiplication" (GETT), a novel approach to tensor contractions that mirrors the design of a high-performance general matrix-matrix multiplication (GEMM). The critical insight behind GETT is the identification of three index sets, involved in the tensor contraction, which enable us to systematically reduce an arbitrary tensor contraction to loops around a highly tuned "macro-kernel". This macro-kernel operates on suitably prepared ("packed") sub-tensors that reside in a specified level of the cache hierarchy. In contrast to previous approaches to tensor contractions, GETT exhibits desirable features such as unit-stride memory accesses, cache-awareness, as well as full vectorization, without requiring auxiliary memory. To compare our technique with other modern tensor contractions, we integrate GETT alongside the so called Transpose-Transpose-GEMM-Transpose and Loops-over-GEMM approaches into an open source "Tensor Contraction Code Generator" (TCCG). The performance results for a wide range of tensor contractions suggest that GETT has the potential of becoming the method of choice: While GETT exhibits excellent performance across the board, its effectiveness for bandwidth-bound tensor contractions is especially impressive, outperforming existing approaches by up to $12.4imes$. More precisely, GETT achieves speedups of up to $1.41imes$ over an equivalent-sized GEMM for bandwidth-bound tensor contractions while attaining up to $91.3\mathrm{\%}$ of peak floating-point performance for compute-bound tensor contractions.

Recommendations

• High-Performance Tensor Contraction without Transposition
• Strassen's Algorithm for Tensor Contraction
• Towards an efficient use of the BLAS library for multilinear tensor contractions
• TTC: a high-performance compiler for tensor transpositions
• Fast bilinear algorithms for symmetric tensor contractions

Cited in

(13)

• Direct reconstruction method for discontinuous Galerkin methods on higher-order mixed-curved meshes III. Code optimization via tensor contraction
• Efficient high-order spectral element discretizations for building block operators of CFD
• A \(\mu\)-mode BLAS approach for multidimensional tensor-structured problems
• Deneb: an open-source high-performance multi-physical flow solver based on high-order DRM-DG method
• Towards an efficient use of the BLAS library for multilinear tensor contractions
• TSFC: A Structure-Preserving Form Compiler
• TTC: a high-performance compiler for tensor transpositions
• High-Performance Tensor Contraction without Transposition
• High performance rearrangement and multiplication routines for sparse tensor arithmetic
• Strassen's Algorithm for Tensor Contraction
• Multidimensional Array Data Management
• Architecture-based and target-oriented algorithm optimization of high-order methods via complete-search tensor contraction
• Algorithm design for tensor units

Describes a project that uses

Uses Software

• TensorToolbox
• NWChem
• P3DFFT
• MADNESS
• MKL
• Algorithm 862
• Algorithm 679
• BLIS
• Tensorlab
• TensorFlow
• TTC
• AUGEM
• cuDNN
• HPTT
• CTF
• fbfft

This page was built for publication: Design of a high-performance GEMM-like tensor-tensor multiplication

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4581393)