Alternating least squares as moving subspace correction
From MaRDI portal
Abstract: In this note we take a new look at the local convergence of alternating optimization methods for low-rank matrices and tensors. Our abstract interpretation as sequential optimization on moving subspaces yields insightful reformulations of some known convergence conditions that focus on the interplay between the contractivity of classical multiplicative Schwarz methods with overlapping subspaces and the curvature of low-rank matrix and tensor manifolds. While the verification of the abstract conditions in concrete scenarios remains open in most cases, we are able to provide an alternative and conceptually simple derivation of the asymptotic convergence rate of the two-sided block power method of numerical algebra for computing the dominant singular subspaces of a rectangular matrix. This method is equivalent to an alternating least squares method applied to a distance function. The theoretical results are illustrated and validated by numerical experiments.
Recommendations
- On global convergence of alternating least squares for tensor approximation
- Local convergence of the alternating least squares algorithm for canonical tensor approximation
- On the global convergence of the alternating least squares method for rank-one approximation to generic tensors
- Some convergence results on the regularized alternating least-squares method for tensor decomposition
- Accurate low-rank approximations via a few iterations of alternating least squares
Cites work
- A new convergence proof for the higher-order power method and generalizations
- Convergence results for projected line-search methods on varieties of low-rank matrices via Łojasiewicz inequality
- Das Verfahren der Treppeniteration und verwandte Verfahren zur Lösung algebraischer Eigenwertprobleme
- scientific article; zbMATH DE number 6159604 (Why is no real title available?)
- scientific article; zbMATH DE number 5223994 (Why is no real title available?)
- scientific article; zbMATH DE number 3280851 (Why is no real title available?)
- scientific article; zbMATH DE number 3381785 (Why is no real title available?)
- Local convergence of the alternating least squares algorithm for canonical tensor approximation
- Low rank matrix completion by alternating steepest descent methods
- Low-rank matrix completion by Riemannian optimization
- Musings on multilinear fitting
- On local convergence of alternating schemes for optimization of convex problems in the tensor train format
- On the global convergence of the alternating least squares method for rank-one approximation to generic tensors
- Preconditioned low-rank Riemannian optimization for linear systems with tensor product structure
- Rank-one approximation to high order tensors
- The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate
- The method of alternating projections and the method of subspace corrections in Hilbert space
Cited in
(10)- Numerical approximation of Poisson problems in long domains
- Alternating least squares in generalized linear models
- Tensor approximation of the self-diffusion matrix of tagged particle processes
- On global convergence of alternating least squares for tensor approximation
- SOTT: greedy approximation of a tensor as a sum of tensor trains
- Accurate low-rank approximations via a few iterations of alternating least squares
- Low-rank tensor methods for partial differential equations
- Computation of the self-diffusion coefficient with low-rank tensor methods: application to the simulation of a cross-diffusion system
- Local convergence of alternating low‐rank optimization methods with overrelaxation
- A low-rank solver for conforming multipatch isogeometric analysis
This page was built for publication: Alternating least squares as moving subspace correction
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4562238)
