A nonlinear GMRES optimization algorithm for canonical tensor decomposition
From MaRDI portal
Abstract: A new algorithm is presented for computing a canonical rank-R tensor approximation that has minimal distance to a given tensor in the Frobenius norm, where the canonical rank-R tensor consists of the sum of R rank-one components. Each iteration of the method consists of three steps. In the first step, a tentative new iterate is generated by a stand-alone one-step process, for which we use alternating least squares (ALS). In the second step, an accelerated iterate is generated by a nonlinear generalized minimal residual (GMRES) approach, recombining previous iterates in an optimal way, and essentially using the stand-alone one-step process as a preconditioner. In particular, the nonlinear extension of GMRES is used that was proposed by Washio and Oosterlee in [ETNA Vol. 15 (2003), pp. 165-185] for nonlinear partial differential equation problems. In the third step, a line search is performed for globalization. The resulting nonlinear GMRES (N-GMRES) optimization algorithm is applied to dense and sparse tensor decomposition test problems. The numerical tests show that ALS accelerated by N-GMRES may significantly outperform both stand-alone ALS and a standard nonlinear conjugate gradient optimization method, especially when highly accurate stationary points are desired for difficult problems. The proposed N-GMRES optimization algorithm is based on general concepts and may be applied to other nonlinear optimization problems.
Recommendations
- Nonlinear least squares solver for evaluating canonical tensor decomposition
- Tensor-GMRES Method for Large Systems of Nonlinear Equations
- A nonlinearly preconditioned conjugate gradient algorithm for rank-R canonical tensor approximation.
- An adaptive algebraic multigrid algorithm for low-rank canonical tensor decomposition
- Generalized canonical polyadic tensor decomposition
- Optimization via separated representations and the canonical tensor decomposition
- A seminorm regularized alternating least squares algorithm for canonical tensor decomposition
- A regularized Newton method for the efficient approximation of tensors represented in the canonical tensor format
- Tensor Methods for Large, Sparse Nonlinear Least Squares Problems
- Optimization-based algorithms for tensor decompositions: canonical polyadic decomposition, decomposition in rank-(L_r,L_r,1) terms, and a new generalization
Cited in
(36)- Alternating cyclic vector extrapolation technique for accelerating nonlinear optimization algorithms and fixed-point mapping applications
- On the asymptotic linear convergence speed of Anderson acceleration, Nesterov acceleration, and nonlinear GMRES
- Nonlinear model order reduction with low rank tensor approximation
- Nonlinear least squares solver for evaluating canonical tensor decomposition
- Non-stationary Anderson acceleration with optimized damping
- News algorithms for tensor decomposition based on a reduced functional
- A nonlinearly preconditioned conjugate gradient algorithm for rank-R canonical tensor approximation.
- On an improved PDE-based elliptic parameterization method for isogeometric analysis using preconditioned Anderson acceleration
- Sparse low-rank separated representation models for learning from data
- The optimization landscape for fitting a rank-2 tensor with a rank-1 tensor
- A fast alternating least squares method for third-order tensors based on a compression procedure
- Nonlinearly preconditioned L-BFGS as an acceleration mechanism for alternating least squares with application to tensor decomposition.
- scientific article; zbMATH DE number 7409363 (Why is no real title available?)
- Nesterov acceleration of alternating least squares for canonical tensor decomposition: momentum step size selection and restart mechanisms.
- On the asymptotic linear convergence speed of Anderson acceleration applied to ADMM
- Convergence properties of nonlinear GMRES applied to linear systems
- Anderson acceleration as a Krylov method with application to convergence analysis
- Total variation based tensor decomposition for multi-dimensional data with time dimension.
- Nonmonotone globalization for Anderson acceleration via adaptive regularization
- Steepest descent preconditioning for nonlinear GMRES optimization
- Accelerating the computation of tensor Z-eigenvalues
- Line search and trust region strategies for canonical decomposition of semi-nonnegative semi-symmetric 3rd order tensors
- Rank properties and computational methods for orthogonal tensor decompositions
- Nonlinearly preconditioned optimization on Grassmann manifolds for computing approximate Tucker tensor decompositions
- A Riemannian trust region method for the canonical tensor rank approximation problem
- An adaptive algebraic multigrid algorithm for low-rank canonical tensor decomposition
- Incremental CP tensor decomposition by alternating minimization method
- A literature survey of low-rank tensor approximation techniques
- Optimization via separated representations and the canonical tensor decomposition
- Some theoretical results on the finite convergence property and the temporary stalling behavior of Anderson acceleration on linear systems
- Range-separated tensor format for many-particle modeling
- Tensor Deflation for CANDECOMP/PARAFAC— Part II: Initialization and Error Analysis
- Convergence analysis of adaptive DIIS algorithms with application to electronic ground state calculations
- Extrapolation methods as nonlinear Krylov subspace methods
- Minimizing finite sums with the stochastic average gradient
- The dynamics of swamps in the canonical tensor approximation problem
This page was built for publication: A nonlinear GMRES optimization algorithm for canonical tensor decomposition
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2909269)
