Normal boundary dilations and rationally invariant subspaces (Q1804026)

For a given Hilbert space operator \(T\) acting on some complex space \(H\), the authors use the following concept of spectral set. A nonvoid, compact subset \(K\) of the complex plane is said to be a spectral set for \(T\) if the norm of \(f(T)\) is less than the supremum of \(| f(\lambda)|\) as \(\lambda\) runs in \(K\), for each \(f\), a rational function with poles off \(K\). The main result is as follows. If \(G\) is a bounded domain in the complex plane, such that its boundary consists of a finite number of Jordan curves, its closure is a spectral set for \(T\), and the boundary of its closure is contained by the spectrum of \(T\), then there is a closed, nontrivial subspace of \(H\) invariated by \(f(T)\) for each rational function \(f\) with poles off the closure of \(G\).