On tensor products of representations of motion groups (Q1097973)

It is an immediate consequence of the orthogonality relations for characters that if \(\pi\) and \(\rho\) are irreducible representations of a compact group G, then the tensor product \(\pi\) \(\otimes {\bar \rho}\) of \(\pi\) with the conjugate of \(\rho\) contains the trivial l-dimensional representation \(l_ G\) if and only if \(\rho =\pi\). When dealing with arbitrary groups and infinite dimensional representations, containment has to be replaced by weak containment. Results of this nature have recently been obtained for various classes of locally compact groups including simply connected nilpotent Lie groups, nilpotent discrete groups and groups with relatively compact conjugacy classes. In this paper motion groups are studied. Thus, let G be a semi-direct product \(G=K\ltimes N\) of a compact connected Lie group K and a (second countable) abelian normal subgroup N, and let \(\pi\) \(\in \hat G\), the set of (equivalence classes of) irreducible representations of G. By Mackey's theory, \(\pi\) is an induced representation \(\pi =ind\) \(G_{G_{\lambda}}(\alpha \lambda)\) where \(\lambda\) \(\in \hat N\), \(G_{\lambda}\) the stability subgroup of \(\lambda\) and \(\alpha \in (G_{\lambda}\cap K)^{\wedge}\). The main and somewhat surprising result is that the following conditions on \(\pi\) turn out to be equivalent (\(\prec\) denotes weak containment): (i) For any representation \(\rho\) of G, \(l_ G\prec \pi \otimes {\bar \rho}\) implies \(\pi\) \(\prec \rho.\) (ii) For every \(\rho\) \(\in \hat G\), \(l_ G\prec \pi \otimes {\bar \rho}\) implies \(\rho =\pi\). (iii) For every closed subgroup H of G and \(\tau\) \(\in \hat H\), \(\tau\) \(\prec \pi | H\) implies \(\pi \prec ind\) \(G_ H\tau.\) (iv) If \(L_{\lambda}\) denotes the largest normal subgroup of K contained in \(K_{\lambda}=G_{\lambda}\cap K\), then \(ind_{L_{\lambda}}^{K_{\lambda}}(\alpha | L_{\lambda})\) is a multiple of \(\alpha\). The proofs are fairly complicated and use detailed knowledge about the dual space topology on \(\hat G\).