On the dimension datum of a subgroup and its application to isospectral manifolds (Q1951487)

For a compact Lie group \(G\) (not necessarily connected) denote by \(\widehat{G}\) its set of irreducible (continuous) representations up to equivalence. Let \(H\) be a closed subgroup of \(G\). The dimension datum \(\mathcal{D}_H:\widehat{G}\to \mathbb{Z},\;V\mapsto\dim V^H\) of \(H\) is the sequence of invariant dimensions of irreducible representations by restriction. First, the authors prove a result proposed by Langlands about the finiteness of the possible \(G\)-conjugacy classes. Theorem 1.2: If the function \(\mathcal{D}_H\) is given, then there are only finitely many possibilities for the conjugacy class of \(H\). Then they obtain some other results concerning the dimension datum. Theorem 1.3: The dimension datum \(\mathcal{D}_H\) determines the cardinality \(|H/H^0|\), the dimension of \(H\), the rank of \(H\), the \(G\)-conjugacy class of the maximal tori of \(H\) and \(\mathcal{D}_{H^0}\), where \(H^0\) is the neutral component of \(H\). In Theorem 1.5, there are obtained examples of connected subgroups \(H_1\) and \(H_2\) such that \(\mathcal{D}_{H_1}=\mathcal{D}_{H_2}\) but \(H_1\), \(H_2\) are not isomorphic. Then it is shown that a certain problem raised by Langlands has a negative answer in general. The Coxeter number of \(H\) is not determined by \(\mathcal{D}_H\), but it follows that \(|h_1-h_2|\leq 1\) if \(\mathcal{D}_{H_1}=\mathcal{D}_{H_2}\), where \(h_i\) is the Coxeter number of \(H^0_i\). Finally, the authors present some applications to isospectral manifolds. They show that, for a compact Lie group (connected and simply connected) and two closed subgroups \(H_1\), \(H_2\) of \(G\) fulfilling some usual conditions, the compact homogeneous manifolds \(G/H_1\) and \(G/H_2\) are isospectral, simply connected and have different homotopy types.