Ratio asymptotics for orthogonal matrix polynomials (Q1125467)

In the 1979's monograph ``Orthogonal polynomials'' [Mem. Am. Math. Soc. 213, 185 p. (1979; Zbl 0405.33009)] \textit{P. Nevai} established a remarkable result that orthogonal polynomials on \(\mathbb R\), satisfying a 3-term recurrence relation with converging coefficients, present a ratio asymptotics outside of the set of accumulation points of the zeros of these polynomials. The limit of the ratio of consecutive polynomials is the Cauchy transform of the weight of orthogonality for Chebyshev polynomials of the second kind. The present paper extends these results to orthogonal matrix polynomials. Namely, if for \(n \geq 0\), \(A_n\) are \(N \times N\) nonsingular matrices and \(B_n\) are \(N \times N\) Hermitian matrices, the recurrence relation \[ t P_n(t)=A_{n+1} P_{n+1}(t) + B_n P_n(t) +A_n^* P_{n-1}(t) , \quad P_{-1}=0 , \quad P_0=I , \] is considered under the assumption that \[ \lim_n A_n =A , \quad \lim_n B_n =B ; \] (in analogy with the scalar case, this setting defines the Nevai class \(M(A,B)\) of matrix polynomials or matrix measures of orthogonality). In the case when \(A\) is nonsingular, it is shown that the sequence \(P_{n-1}(z) P_n^{-1}(z)A_n^{-1}\) converges to a matrix-valued function \(F_{A,B}(z)\) outside of the set \(\Gamma\) of accumulation points of zeros of \(\det (P_n)\). The function \(F_{A,B}\) is in fact the Cauchy transform of the positive definite matrix of measures \(W_{A,B}\) which generalizes the orthogonality weight for Chebyshev polynomials of the second kind. The author finds explicit expressions for \(F_{A,B}\) (when \(A\) is Hermitian) and for \(W_{A,B}\) (when \(A\) is positive definite). Nevertheless, if \(A\) is just nonsingular, many questions about both \(F_{A,B}\) and \(W_{A,B}\) remain open. The last section of the paper is devoted to the degenerate case, that is, when \(A\) is singular. Then, the sequence \(P_{n-1}(z) P_n^{-1}(z)A_n^{-1}\) still converges to \(F_{A,B}(z)\) in \(\mathbb C \setminus \Gamma\), and \(F_{A,B}\) is the Cauchy transform of a positive definite matrix of measures \(\nu\), but now, generally speaking, \(\nu \neq W_{A,B}\). Moreover, \(\nu\) cannot generate a sequence of orthogonal matrix polynomials. The author completes the paper discussing different forms of degeneracy of \(\nu\) when \(A\) is singular.