Block similarity invariants of restrictions to \((A,B)\)-invariant subspaces (Q1893080)

Let be given a pair of matrices \((A, B)\) over a commutative field \(F\), where \(A\) is \(p\times p\) and \(B\) is \(q\times p\). Block similarity invariants of \((A, B)\) can be identified with the Kronecker invariants of the singular pencil \[ \lambda\biggl[\begin{smallmatrix} I_ p\\ 0\end{smallmatrix}\biggr]- \biggl[\begin{smallmatrix} A\\ B\end{smallmatrix}\biggr], \] i.e., the row-minimal indices and the invariant factors of the pencil. A complete characterization is obtained (in terms of divisibility relations) of the possible block similarity invariants of pairs of matrices of the form \[ \biggl(\biggl[\begin{smallmatrix} A_ 1\\ 0\end{smallmatrix} \begin{smallmatrix} A_ 3\\ A_ 2\end{smallmatrix}\biggr],\;\biggl[\begin{smallmatrix} B_ 1\\ 0\end{smallmatrix} \begin{smallmatrix} B_ 3\\ B_ 2\end{smallmatrix}\biggr]\biggr), \] where \((A_ 1, B_ 1)\) is a pair of matrices with given block similarity invariants. The proof is based on interlacing inequalities for invariant factors of polynomial matrices, canonical form of matrix pairs under block similarity, and the well-known connections between the invariant polynomials of a linear transformation in a finite-dimensional vector space and those of the restriction of the linear transformation to its invariant subspace. The main result is also presented in a basis free formulation, using the notion of invariant subspaces for linear transformations between two different finite-dimensional vector spaces.