Selfadjoint operators in S-spaces (Q629697)

In previous papers by one of the authors, the so-called S-spaces were introduced, generalizing the concept of Krein spaces: a Hilbert space \((\mathfrak S, (\cdot,-))\) equipped with an additional (not necessarily hermitian) inner product \([f,g]=(Uf,g)\), where \(U\) is an unitary operator in \((\mathfrak S, (\cdot,-))\). Then, of course, the spectrum of \(U\) lies on the unit circle. If, additionally, \(U\) is selfadjoint, then \((\mathfrak S, [\cdot,-])\) is a Krein space with fundamental symmetry \(U\) and \(\sigma (U) = \{-1, 1\}\). Conversely, using a fundamental symmetry as \(U\), each Krein space is also an S-space. As to the knowledge of the reviewer, there was a certain discussion whether these S-spaces are helpful at all. In fact, it is shown in the present paper that the concepts of selfadjoint operators coincide for Krein and S-spaces (with a ``natural'' definition of selfadjointness): Starting with a selfadjoint operator \(A\) in an S-space \((\mathfrak S, [\cdot,-])\), it is possible to construct a Krein space inner product on \(\mathfrak S\) such that \(A\) is also selfadjoint in this Krein space. However, this new concept allows a new approach to the problem of \(A\)-invariant subspaces which is a well-known (and nontrivial) problem for selfadjoint operators in Krein spaces without additional conditions (e.g., definitizability). The paper presents the following result: Take a Borel subset \(\Delta\) of the unit circle, symmetric with respect to the origin (i.e., \(-\Delta = \Delta\)) and the spectral measure \(E_U\) of \(U\). Then \(E_U(\Delta) \mathfrak S\) is an invariant subspace of any selfadjoint operator \(A\) in \((\mathfrak S, [\cdot,-])\). In particular, if \(\sigma (U) \neq \{-1, 1\}\), we may obtain a large number of nontrivial invariant subspaces. It remains an open question which class of selfadjoint operators in Krein spaces are covered by this approach with \(\sigma (U) \neq \{-1, 1\}\).