Parallel Algorithms for Tensor Train Arithmetic
From MaRDI portal
Abstract: We present efficient and scalable parallel algorithms for performing mathematical operations for low-rank tensors represented in the tensor train (TT) format. We consider algorithms for addition, elementwise multiplication, computing norms and inner products, orthogonalization, and rounding (rank truncation). These are the kernel operations for applications such as iterative Krylov solvers that exploit the TT structure. The parallel algorithms are designed for distributed-memory computation, and we use a data distribution and strategy that parallelizes computations for individual cores within the TT format. We analyze the computation and communication costs of the proposed algorithms to show their scalability, and we present numerical experiments that demonstrate their efficiency on both shared-memory and distributed-memory parallel systems. For example, we observe better single-core performance than the existing MATLAB TT-Toolbox in rounding a 2GB TT tensor, and our implementation achieves a speedup using all 40 cores of a single node. We also show nearly linear parallel scaling on larger TT tensors up to over 10,000 cores for all mathematical operations.
Recommendations
- Parallel Algorithms for Computing the Tensor-Train Decomposition
- Parallel Algorithms for Low Rank Tensor Arithmetic
- Parallel approximation of multidimensional tensors using GPUs
- Efficient vector and parallel manipulation of tensor products
- High performance rearrangement and multiplication routines for sparse tensor arithmetic
- scientific article; zbMATH DE number 4176333
- Parallel Algorithms for Dense Linear Algebra Computations
- scientific article; zbMATH DE number 47823
- Parallel algorithms for certain matrix computations
- scientific article; zbMATH DE number 193901
Cites work
- \(O(d \log N)\)-quantics approximation of \(N\)-\(d\) tensors in high-dimensional numerical modeling
- A new scheme for the tensor representation
- A survey of projection-based model reduction methods for parametric dynamical systems
- An overview of the Trilinos project
- Analysis of individual differences in multidimensional scaling via an \(n\)-way generalization of ``Eckart-Young decomposition
- Block tensor unfoldings
- Certified reduced basis methods for parametrized partial differential equations
- Communication lower bounds and optimal algorithms for numerical linear algebra
- Communication-optimal parallel and sequential QR and LU factorizations
- Faster tensor train decomposition for sparse data
- Krylov subspace methods for linear systems with tensor product structure
- Low-rank solution to an optimization problem constrained by the Navier-Stokes equations
- Low-rank solvers for unsteady Stokes-Brinkman optimal control problem with random data
- Low-Rank Tensor Approximation for High-Order Correlation Functions of Gaussian Random Fields
- Numerical operator calculus in higher dimensions
- PLANC
- Recompression of Hadamard Products of Tensors in Tucker Format
- Reduced basis methods for partial differential equations. An introduction
- Tensor approximations of matrices generated by asymptotically smooth functions
- Tensor Decompositions and Applications
- Tensor train approximation of moment equations for elliptic equations with lognormal coefficient
- Tensor-train decomposition
- TT-cross approximation for multidimensional arrays
- TuckerMPI: a parallel C++/MPI software package for large-scale data compression via the Tucker tensor decomposition
Cited in
(17)- Adaptive integration of nonlinear evolution equations on tensor manifolds
- Parallel ALS algorithm for solving linear systems in the hierarchical Tucker representation
- Distributed hierarchical SVD in the hierarchical Tucker format.
- Efficient vector and parallel manipulation of tensor products
- Performance of the low-rank TT-SVD for large dense tensors on modern multicore CPUs
- Parallel Algorithms for Low Rank Tensor Arithmetic
- Higher-Order QR with Tournament Pivoting for Tensor Compression
- Randomized Algorithms for Rounding in the Tensor-Train Format
- Generative modeling via tensor train sketching
- Implicit integration of nonlinear evolution equations on tensor manifolds
- Parallel approximation of multidimensional tensors using GPUs
- Tensor rank reduction via coordinate flows
- Parallel Algorithms for Computing the Tensor-Train Decomposition
- Imposing different boundary conditions for thermal computational homogenization problems with FFT- and tensor-train-based Green's operator methods
- Efficient randomized algorithms for computing an approximation of the tensor train decomposition
- APTT: accuracy-preserved tensor-train method for the Boltzmann-BGK equation
- Fundamental tensor operations for large-scale data analysis using tensor network formats
Describes a project that uses
Uses Software
This page was built for publication: Parallel Algorithms for Tensor Train Arithmetic
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5028405)
