On generalized resolvents of Hermitian relations in Krein spaces (Q1971743)

Hermitian and selfadjoint relations and their generalized resolvents have been studied in various papers [see e. g. \textit{V. Derkach}, Integral Equations Oper. Theory 23, No. 4, 387-415 (1995; Zbl 0837.47030) and \textit{A. Dijksma, H. Langer} and \textit{H. de Snoo}, Lect. Notes Math. 1242, 1-42 (1987; Zbl 0628.47026); Oper. Theory, Adv. Appl. 48, 261-274 (1990; Zbl 0734.47018)]. This paper is an important continuation of that work. Let \(A\) be a closed Hermitian linear relation on a Krein space \(\mathcal K\), i. e. \(A\subset A^+\), where \(A^+\) is the Krein space adjoint of \(A\). A triple \(\Pi=\{{\mathcal H}, \Gamma _1,\Gamma _2\}\) is called a boundary-value space (BVS) of \(A^+\) if \(\mathcal H\) is a Hilbert space, \(\Gamma =(\Gamma _1,\Gamma_2)\) is a bounded operator from \(A^+\) onto \({\mathcal H}\oplus{\mathcal H}\), and \[ [f',g]-[f,g']=(\Gamma _1 \widehat f,\Gamma _2 \widehat g)_{\mathcal H} -(\Gamma _2 \widehat f,\Gamma _1 \widehat g)_{\mathcal H} \] for all \(\widehat f=\{f,f'\}.\widehat g=\{g,g'\}\in A^+\). There is a one-to-one relation between extensions \(\widetilde A\) of \(A\), \(A\subset \widetilde A\subset A^+\), and subspaces \(\theta\) of \({\mathcal H}\oplus{\mathcal H}\) given by \(\theta=\Gamma \widetilde A\), \(\widetilde A=\widetilde A_\theta=\Gamma ^{-1}\theta\). Each BVS has two natural selfadjoint extensions \(A_i=\text{ker}\Gamma _i\) (\(i=1,2\)). Let \({\mathcal N}_\lambda={\mathcal K}[-]{\mathcal R}(A-\overline{\lambda })\), \(\widehat{\mathcal N}_\lambda=\{\{f_\lambda ,\lambda f_\lambda \}\mid f_\lambda \in {\mathcal N}_\lambda\}\). The operator-valued function \(M(\lambda)\) defined on \(\rho(A_2)\) by \(M(\lambda)\Gamma _2\widehat f_\lambda =\Gamma _1 f_\lambda \), \(\lambda \in \rho(A_2)\), \(\widehat f_\lambda \in \widehat{\mathcal N}_\lambda \), is called the Weyl function of \(A\) corresponding to the BVS \(\Pi=\{{\mathcal H}, \Gamma _1,\Gamma _2\}\). The Weyl function contains all information about the spectrum of an extension \(\widetilde A_\theta \) of \(A\) outside \(\rho(A_2)\): \(\lambda \in \tau(\widetilde A_\theta)\Leftrightarrow \theta \in \tau(M(\lambda)-\theta)\), where \(\tau\) is any of \(\sigma _p\), \(\sigma _r\), \(\rho\), and \(\text{ker}(\widetilde A_\theta-\lambda) =\gamma (\lambda)\text{ker}(M(\lambda)-\theta)\) for \(\lambda \in \sigma _p(\widetilde A_\theta)\), \[ (\widetilde A_\theta-\lambda)^{-1}=(A_2-\lambda)^{-1}+ \gamma (\lambda)(\theta-M(\lambda))^{-1}\gamma ^+(\overline{\lambda }) \;(\lambda \in\rho(\widetilde A_\theta)\cap \rho(A_2)) \] where \(\gamma(\lambda)=\pi_1(\Gamma _2|\widehat{\mathcal N}_\lambda)^{-1}\), Theorem 2.1. If, additionally, \(A_2\) is definitizable and \(\infty \) not a critical singular point of \(A_2\), then \[ M(\lambda)=\sum_{j=1}^\nu M_j\lambda ^j+M_0(\lambda) \] with selfadjoint operators \(M_j\in L({\mathcal H})\), and \(M_0\) is a holomorphic function from \(\rho(A_2)\) into \(L({\mathcal H})\) satisfying \[ \lim_{y\uparrow \infty }\frac{M_0(iy)}y=0. \] The main part of the paper deals with generalized resolvents. Recall that a generalized resolvent of \(A\) is a holomorphic function \(R_\lambda \) from a region \(\mathcal O\) into \(L({\mathcal K})\) such that there is a Krein space \({\mathcal K'}\) and a self-adjoint extension \(\widetilde A\) of \(A\) in the Krein space \(\widetilde{\mathcal K}={\mathcal K}\oplus{\mathcal K}\) such that \(R_\lambda =P_{\mathcal K}(\widetilde A-\lambda)^{-1}|_{\mathcal K}\) for \(\lambda \in\rho(\widetilde A)\cap {\mathcal O}\neq \emptyset \), shortly \(R_\lambda \in P\Omega (\zeta)\) for \(\zeta\in \rho(\widetilde A)\cap {\mathcal O}\). Special attention is given to the case that \({\mathcal K}'\) is a Pontryagin space \(\Pi_\kappa \) and the extension is minimal. In Theorem 4.1 it is shown that \(R_\lambda =(A_2-\lambda)^{-1}+\gamma (\lambda)\phi(\lambda)\gamma ^+ (\overline{\lambda })\) establishes a one-to-one correspondence between \({\mathcal P}\Omega (\rho)\) and the set of all functions \(\phi(\lambda)\) holomorphic at \(\zeta \) with values in \(L({\mathcal H})\). The self-adjoint realization is given by a boundary value problem with the spectral parameter in the boundary condition: \(f={\mathcal R}_\lambda h\) if and only if \(\{f,h\}\in A^+-\lambda \), \((I+\phi(\lambda)M(\lambda))\Gamma _2\widehat f-\phi(\lambda)\Gamma _1\widehat f=0\), where \(\widehat f=\{f,h+\lambda f\}\in A^+\). Finally, in Section 6 the truncated moment problem in Krein spaces is investigated.