Spaces invariant under unitary representations of discrete groups (Q2657663)

This paper introduces general tools for the study of closed subspaces of a Hilbert space that are invariant under a unitary representation of a countable group on that Hilbert space. Let \(\Gamma\) be a countable group and \(\Pi\) a unitary representation of \(\Gamma\) on a separable Hilbert space \(\mathcal{H}\). There are a variety of known results concerning \(\Pi\)-invariant closed subspaces of \(\mathcal{H}\) when \(\Gamma\) is abelian. The authors formulate and prove generalizations of many of these results to non-abelian \(\Gamma\). In these generalizations, \(\mathcal{R}(\Gamma)\) plays the role of the dual of \(\Gamma\), where \(\mathcal{R}(\Gamma)\) is the von Neumann algebra generated by the right regular representation \(\rho\) of \(\Gamma\). The usual Lebesgue spaces on the dual of \(\Gamma\) are replaced by the noncommutative \(L^p\)-spaces formed using the canonical finite trace \(\tau\) on \(\mathcal{R}(\Gamma)\). The significant results in this paper assume that \(\Pi\) satisfies the square-integrability condition that there exists a dense subspace \(\mathcal{D}\) of \(\mathcal{H}\) such that \(x\mapsto \langle\varphi,\Pi(x)\psi\rangle\) is in \(\ell^2(\Gamma)\), for all \(\varphi\in\mathcal{H}\) and \(\psi\in\mathcal{D}\). In that case, there exists a sesquilinear map \([\cdot,\cdot]: \mathcal{H}\times\mathcal{H}\to L^1\big(\mathcal{R}(\Gamma)\big)\) which satisfies \(\langle\varphi,\Pi(x)\psi\rangle= \tau\big([\varphi,\psi]\rho(x)\big)\), for all \(\varphi,\psi\in\mathcal{H}\) and \(x\in\Gamma\). The authors refer to any triple \((\Gamma,\Pi,\mathcal{H})\) as being dual integrable if such a bracket map \([\cdot,\cdot]\) exists. A core theorem is that \((\Gamma,\Pi,\mathcal{H})\) is dual integrable if and only if there exists a \(\sigma\)-finite measure space \((\mathcal{M},\nu)\) and an isometry \(\mathcal{J}:\mathcal{H}\to L^2\big((\mathcal{M},\nu),L^2(\mathcal{R}(\Gamma))\big)\) that satisfies \(\mathcal{J}\big(\Pi(x)\varphi\big)= \mathcal{J}(\varphi)\rho(x)^*\), for all \(x\in\Gamma\) and \(\varphi\in\mathcal{H}\). This map \(\mathcal{J}\) is called a Helson map as it directly generalizes the Helson map used in characterizing shift-invariant closed subspaces of \(L^2(\mathbb{R})\). This generalized Helson map is the main tool in generalizing results on invariant subspaces to nonabelian \(\Gamma\).