On irreducibility of an induced representation of a simply connected nilpotent Lie group (Q907862)

Let \(G\) be a simply connected nilpotent Lie group with Lie algebra \(\mathfrak g\), \(V\) a finite dimensional \(\mathbb{K}\)-vector space (\(\mathbb{K}=\mathbb{R}\), or \(\mathbb{C}\)), \(\pi: \mathfrak g\to \mathfrak{gl}(V)\) a representation of \(\mathfrak g\), where \(\mathfrak{gl}(V)\) is the Lie algebra of endomorphisms of \(V\). Let \(\mathfrak h\) be a Lie subalgebra of \(\mathfrak g\) subordinate to \(\pi\) (that is, \(\pi([\mathfrak h, \mathfrak h])=\{0\}\)) and let \(H\) be the connected Lie subgroup of \(G\) associated to \(\mathfrak h\). We define a representation \(\chi_\pi\) of \(H\) in the space \(V\) by the rule \(\chi_\pi(\exp X) =e^{i\pi(X)}\) for every \(X\in \mathfrak h\). Let \(\rho(\pi, H, G)= \mathrm{ind}_H^G\chi_\pi\) be the unitary representation of \(G\) induced by the representation \(\chi_\pi\) of \(H\). In the paper under review, the authors study the irreducibility of the representation \(\rho(\pi, H, G)\) in the setting of simply connected nilpotent Lie groups \(G\). More precisely, the authors prove that for a non-abelian simply connected nilpotent Lie group \(G\) such that \(\mathcal{Z}\cap \ker(\pi)\neq\{0\}\), where \(\mathcal{Z}\) denotes the center of the Lie algebra \(\mathfrak g\), the representation \(\rho(\pi, H, G)\) is irreducible if and only if \(\mathfrak h\) is a polarization at \(\pi\) (Theorem 3.3).