Parallel Algorithms for Tensor Train Arithmetic
From MaRDI portal
Publication:5028405
DOI10.1137/20M1387158zbMATH Open1484.65088arXiv2011.06532OpenAlexW4210523970WikidataQ115214713 ScholiaQ115214713MaRDI QIDQ5028405FDOQ5028405
Authors: Hussam al Daas, Grey Ballard, P. Benner 
Publication date: 9 February 2022
Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)
Abstract: We present efficient and scalable parallel algorithms for performing mathematical operations for low-rank tensors represented in the tensor train (TT) format. We consider algorithms for addition, elementwise multiplication, computing norms and inner products, orthogonalization, and rounding (rank truncation). These are the kernel operations for applications such as iterative Krylov solvers that exploit the TT structure. The parallel algorithms are designed for distributed-memory computation, and we use a data distribution and strategy that parallelizes computations for individual cores within the TT format. We analyze the computation and communication costs of the proposed algorithms to show their scalability, and we present numerical experiments that demonstrate their efficiency on both shared-memory and distributed-memory parallel systems. For example, we observe better single-core performance than the existing MATLAB TT-Toolbox in rounding a 2GB TT tensor, and our implementation achieves a speedup using all 40 cores of a single node. We also show nearly linear parallel scaling on larger TT tensors up to over 10,000 cores for all mathematical operations.
Full work available at URL: https://arxiv.org/abs/2011.06532
Recommendations
- Parallel Algorithms for Computing the Tensor-Train Decomposition
- Parallel Algorithms for Low Rank Tensor Arithmetic
- Parallel approximation of multidimensional tensors using GPUs
- Efficient vector and parallel manipulation of tensor products
- High performance rearrangement and multiplication routines for sparse tensor arithmetic
- scientific article; zbMATH DE number 4176333
- Parallel Algorithms for Dense Linear Algebra Computations
- scientific article; zbMATH DE number 47823
- Parallel algorithms for certain matrix computations
- scientific article; zbMATH DE number 193901
Parallel numerical computation (65Y05) Complexity and performance of numerical algorithms (65Y20) Factorization of matrices (15A23) Multilinear algebra, tensor calculus (15A69) Numerical linear algebra (65F99)
Cites Work
- Reduced basis methods for partial differential equations. An introduction
- TuckerMPI: a parallel C++/MPI software package for large-scale data compression via the Tucker tensor decomposition
- Analysis of individual differences in multidimensional scaling via an \(n\)-way generalization of ``Eckart-Young decomposition
- Tensor Decompositions and Applications
- Tensor-train decomposition
- TT-cross approximation for multidimensional arrays
- Communication-optimal parallel and sequential QR and LU factorizations
- Numerical operator calculus in higher dimensions
- Tensor approximations of matrices generated by asymptotically smooth functions
- A survey of projection-based model reduction methods for parametric dynamical systems
- Krylov subspace methods for linear systems with tensor product structure
- An overview of the Trilinos project
- A new scheme for the tensor representation
- \(O(d \log N)\)-quantics approximation of \(N\)-\(d\) tensors in high-dimensional numerical modeling
- Low-Rank Tensor Approximation for High-Order Correlation Functions of Gaussian Random Fields
- Certified reduced basis methods for parametrized partial differential equations
- Block tensor unfoldings
- Low-rank solvers for unsteady Stokes-Brinkman optimal control problem with random data
- Recompression of Hadamard Products of Tensors in Tucker Format
- PLANC
- Communication lower bounds and optimal algorithms for numerical linear algebra
- Low-rank solution to an optimization problem constrained by the Navier-Stokes equations
- Faster tensor train decomposition for sparse data
- Tensor train approximation of moment equations for elliptic equations with lognormal coefficient
Cited In (15)
- Parallel ALS algorithm for solving linear systems in the hierarchical Tucker representation
- Parallel Algorithms for Low Rank Tensor Arithmetic
- Generative modeling via tensor train sketching
- Implicit integration of nonlinear evolution equations on tensor manifolds
- Higher-Order QR with Tournament Pivoting for Tensor Compression
- Parallel approximation of multidimensional tensors using GPUs
- Performance of the low-rank TT-SVD for large dense tensors on modern multicore CPUs
- Parallel Algorithms for Computing the Tensor-Train Decomposition
- Adaptive integration of nonlinear evolution equations on tensor manifolds
- Randomized Algorithms for Rounding in the Tensor-Train Format
- Tensor rank reduction via coordinate flows
- Fundamental tensor operations for large-scale data analysis using tensor network formats
- Imposing different boundary conditions for thermal computational homogenization problems with FFT- and tensor-train-based Green's operator methods
- Distributed hierarchical SVD in the hierarchical Tucker format.
- Efficient vector and parallel manipulation of tensor products
Uses Software
This page was built for publication: Parallel Algorithms for Tensor Train Arithmetic
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5028405)
