Extrema of sums of heterogeneous quadratic forms
This big paper focuses on the following problem arising in various situations in multivariate statistical analysis. Find the maximum of the quadratic functional \[ f_{ \text{quad}} (X)=\sum^K_{i=1} x^T_iA_ix_i \] under the constraint that the vectors \(x_1, \dots, x_k\in \mathbb{R}^n\) form an orthonormal system where \(A_1, \dots, A_k\) \((k\leq n)\) are given \(k\) symmetric positive definite \(n\times n\) matrices and the \(n\times k\) matrix \(X=[x_1, \dots, x_k]\) contains the enumerated vectors as their columns. The set of orthonormal \(k\)-tuples in \(\mathbb{R}^n\) is called a Stiefel manifold and denoted by \(V_{n,k}\); \(X\in V_{n,k}\) when the set of column vectors of the \(n\times k\) matrix \(X\) is an element of \(V_{n,k}\). As \(V_{n,k}\) is a compact manifold and \(f_{\text{quad}}\) is continuous on \(V_{n,k}\) a finite global maximum exists and it is attained at some point. To characterize the critical points of the functional let us denote by \(A(X)= [A_1x_1, \dots, A_kx_k]\) the \(n\times k\) matrix. The main result is the following theorem. Theorem 3.1. \(X\in V_{n,k}\) is a critical point of \(f_{\text{quad}}\) if and only if \(A(X)=XS\), where \(S\) is a symmetric \(k\times k\) matrix. At an arbitrary critical point the matrix \(S=X^TA(X)\) is not necessarily positive semidefinite. The next lemma shows that this is necessary at the global maximum points of \(f_{\text{quad}}\). Lemma 3.1. If \(X\in V_{n,k}\) is a global maximum of the functional \(f_{\text{quad}}\), then the corresponding matrix \(S=X^T A(X)\) is positive semidefinite. Theorem 3.1 and Lemma 3.1 together yield that at global maximum points \(A(X)=XS\), where \(S\) is a positive semidefinite matrix. Beyond it a computational iteration algorithm is proposed. Choosing an arbitrary initial orthonormal system \(X^{(0)}= [x_1^{(0)}, \dots, x_k^{(0)}]\in V_{n,k}\), the sequence \(X^{(1)}, X^{(2)}, \dots\) in \(V_{n,k}\) is constructed in the following way: from the \(m\)th element of this sequence \(X^{(m)} =[x_1^{(m)}, \dots, x_k^{(m)}]\in V_{n,k}\) the next one is obtained by a polar decomposition of the matrix \(A(X^{(m)})\) as \(A(X^{(m)})= X^{(m+1)} S^{(m+1)}\) \((m=0,1, \dots,)\) where \((X^{(m+1)})^T X^{(m+1)} =I_k\) and \(S^{(m+1)}\geq 0\). This polar decomposition is unique if \(A_1x_1^{(m)}, \dots, A_kx_k^{(m)}\) are linearly independent. It is proved that \(f_{\text{quad}} (X^{(m)})\) is a nondecreasing sequence and \(\text{dist} (X^{(m)}, {\mathcal C})\to 0\) as \(m\to\infty\) where \({\mathcal C}= \{X\in V_{n,k}:X\) is a critical point of \(f_{\text{quad}}\}\). In the case when \(f_{\text{quad}}\) has isolated critical points the algorithm converges to one of the critical points. The critical points of the quadratic functional assuming some relations between the matrices \(A_1, \dots, A_k\) are considered. Structural properties of the functionals \(f_{\text{quad}}\) and \[ f_{\text{bilin}} (X,Y)= \sum^k_{i=1} y^T_iA_ix_i \] where \(x_1, \dots, x_k\) and \(y_1, \dots, y_k\) are orthonormal vectors are discussed.
- Maximization of sums of quotients of quadratic forms and some generalizations
- scientific article; zbMATH DE number 1383079
- scientific article; zbMATH DE number 4183220
- scientific article; zbMATH DE number 4052225
- scientific article; zbMATH DE number 5049113
- Extrema of quadratic forms and statistical applications
- scientific article; zbMATH DE number 4144053
- Extremal Problems and Quadratic Differentials
- Inequalities for sums of matrix quadratic forms
- scientific article; zbMATH DE number 3854298 (Why is no real title available?)
- scientific article; zbMATH DE number 3714378 (Why is no real title available?)
- scientific article; zbMATH DE number 3527527 (Why is no real title available?)
- Least squares matching problems
- Least-Squares Estimation, Linear Programming, and Momentum: A Geometric Parametrization of Local Minima
- Nondegenerate critical manifolds
- The geometry of matrix eigenvalue methods
- On the lower bound for a quadratic problem on the Stiefel manifold
- Some global optimization problems on Stiefel manifolds
- On minimization on Stiefel manifolds
- Maximization of sums of quotients of quadratic forms and some generalizations
- First order optimality conditions and steepest descent algorithm on orthogonal Stiefel manifolds
- Maximizing sum of coupled traces with applications
- Exact solutions to \({\displaystyle\max_{\Vert x\Vert=1}\sum_{i=1}^\infty\Vert T_i(x) \Vert^2}\) with applications to physics, bioengineering and statistics
- Quadratic optimization with orthogonality constraint: explicit Łojasiewicz exponent and linear convergence of retraction-based line-search and stochastic variance-reduced gradient methods
- Anisotropic orthogonal Procrustes analysis
- A note on simultaneously diagonalizable matrices
- scientific article; zbMATH DE number 1795207 (Why is no real title available?)
- Orthogonal trace-sum maximization: applications, local algorithms, and global optimality
- On the local maxima of a constrained quadratic form
- scientific article; zbMATH DE number 5049113 (Why is no real title available?)
- Eine Abschätzung für quadratische Formen
- Matrix-variate data analysis by two-way factor model with replicated observations
- Some new results on orthogonally constrained Candecomp
- A theory of the NEPv approach for optimization on the Stiefel manifold
- Quasi Maximum Likelihood Estimation for Large-Dimensional Matrix Factor Models
- Accelerated gradient methods on the Grassmann and Stiefel manifolds
- A semidefinite relaxation for sums of heterogeneous quadratic forms on the Stiefel manifold
- Improved approximation algorithms for orthogonally constrained problems using semidefinite optimization
- A framework of constraint preserving update schemes for optimization on Stiefel manifold
- On extremum properties of orthogonal quotients matrices
- Maximization of the sum of the trace ratio on the Stiefel manifold. II: computation
This page was built for publication: Extrema of sums of heterogeneous quadratic forms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1375540)
