Pairs of matrices which do not admit a complementary triangular form (Q1174688)

A pair \(A\), \(B\) of \(n\) by \(n\) matrices is said to admit a complementary triangular form if there exists a similarity transform such that \(XAX^{-1}\) and \(XBX^{-1}\) are respectively upper and lower triangular. Such a similarity exists if at least one of the matrices \(A\), \(B\) is diagonalizable. Denote by \(\Delta^ 0_ n\) the set of all \(n\) by \(n\) matrices which leave \(n - 1\) distinct eigenvalues and are not diagonalizable (hence each eigenvalue has exactly one eigenvector). The main result: suppose \(A\), \(B \in \Delta^ 0_ n\). Then a necessary and sufficient condition for \((A, B)\) not to admit a complementary triangular form is given (in terms of the eigenvectors corresponding to the multiple eigenvalues of \(A\) and \(B\)).