Maximal \(\mathcal{J}\)-semi-definite invariant subspaces of unbounded \(\mathcal{J}\)-selfadjoint operators in Krein spaces (Q2227566)

As long as people have been studying \({\mathcal J}\)-selfadjoint operators in Krein spaces, they have also been looking for maximal \({\mathcal J}\)-non-negative and \({\mathcal J}\)-non-positive invariant subspaces. Since the middle of the 20th century, a~number of criteria for its existence have been found. The present paper presents a new approach to existence conditions in the case when the \({\mathcal J}\)-selfadjoint operator \({\mathcal A}\) is the closure of a \({\mathcal J}\)-symmetric operator \({\mathcal A}_0\) which allows a matrix representation \({\mathcal A}_0 = \left(\begin{smallmatrix} A & B \\ -B^* & D \end{smallmatrix}\right)\) with respect to the canonical Krein space decomposition \({\mathcal H} = {\mathcal H}_+ \oplus {\mathcal H}_-\) and the fundamental symmetry \({\mathcal J} = \operatorname{diag}(I_{{\mathcal H}_+}, -I_{{\mathcal H}_-})\). In this setting, the previously known criteria include essential restrictions (e.g., boundedness of some entries in the matrix) which are not required here. Existence conditions are obtained in three different cases. The most general case (i.e., \(\operatorname{dom} D \subset \operatorname{dom} B\) or \(\operatorname{dom} A \subset \operatorname{dom} B^*\)) is reduced to Krein's result by means of the Cayley transform. This leads to the result in the so-called diagonally dominant case (i.e., \(\operatorname{dom} D \subset \operatorname{dom} B\) and \(\operatorname{dom} A \subset \operatorname{dom} B^*\)). Then, if \(B\) is \(D\)-compact or \(B^*\) is \(A\)-compact, the existence of both \({\mathcal A}\)-invariant subspaces is proved: a~maximal \({\mathcal J}\)-non-negative subspace \({\mathcal L}_+\) and a maximal \({\mathcal J}\)-non-positive subspace \({\mathcal L}_-\) (\(= {\mathcal L}_+^{[\bot]}\)). Furtermore, an observation on the non-real part of the spectrum of \({\mathcal A}\) is presented: it may only accumulate at \(\sigma_e(A) \cap \sigma_e(D)\) (which implies no finite accumulation point if \(A\) or \(D\) has compact resolvent). The last case is the so-called corner dominant case (i.e., \(\operatorname{dom} |A|^{\frac{1}{2}} \subset \operatorname{dom} B^*\) or \(\operatorname{dom} |D|^{\frac{1}{2}} \subset \operatorname{dom} B\)). Then, a similar existence statement is obtained as in the diagonally dominant case but only for one maximal \({\mathcal J}\)-semidefinite invariant subspace and an accumulation statement is obtained only for \(\sigma_e(A)\) or \(\sigma_e(D)\), respectively. Finally, the diagonally dominant result is applied to some dissipative two-channel Hamiltonians.