Limit formulas of period integrals for a certain symmetric pair. II (Q719094)

Let \(G\) be a real semisimple Lie group and \(\hat G\) the set of equivalence classes of irreducible representations of \(G\). A well known problem in harmonic analysis is the following: If \(\pi\) is a representation on the space \(V\), how can \(\pi\) be expressed in terms of \(\hat G\)? Consider the case where \(\Gamma\subset G\) is a co-compact lattice of \(G\), \(V = L^2(\Gamma\backslash G)\) and \(\pi=R_{\Gamma}\) the right regular representation of \(G\) on \(V\). One can show that \[ R_{\Gamma} = \sum_{\omega \in \hat G} m(\Gamma,\omega)\omega, \] where \(m(\Gamma,\omega)\) is the multiplicity of \(\omega\) in the right regular representation (see [\textit{I. M. Gelfand, M. I. Graev} and \textit{I. I. Piatetski-Shapiro}, Representation theory and automorphic functions (Russian). Moskau: Verlag 'Nauka' (1966; Zbl 0138.07201); Transl. from the Russian by K.A. Hirsch. Reprint. Boston, MA etc.: Academic Press, Inc. (1990; Zbl 0718.11022)]). The knowledge of these multiplicities gives a complete answer to the posed problem above. \textit{D. L. DeGeorge} and \textit{N. R. Wallach} showed a theorem which does not give a closed formula for \(m(\Gamma,\omega)\) but provides information about these multiplicities [Ann. Math. (2) 107, 133--150 (1978; Zbl 0397.22007); ibid. 109, 477--495 (1979; Zbl 0482.43006)]. \textbf{Theorem 1.1.} Let \(\Gamma= \Gamma_1 \subset \Gamma_2 \subset \cdots \Gamma_n\subset \cdots\) be a tower of normal subgroups of finite index with \(\bigcap_{j=1}^{\infty}\Gamma_j = \{e\}\). Let \(\mu\) be the Plancherel measure on \(\hat G\). For \(S \subset G\) open and relatively compact (regular for the Plancherel measure) define the measure \[ \mu_{\Gamma}(S) = \frac{1}{\text{vol}(\Gamma\backslash G)}\sum_{\omega\in S}m(\Gamma,\omega). \] Then \[ \lim_{j\to \infty} \mu_{\Gamma_j}(S) = \mu(S). \] There are various generalizations of this theorem in the literature. The paper under review is the second part of two papers which deal with the following related problem: ``Is there any measure \(\mu_{\Gamma}^{H}\) on the '\(H\)-spherical unitary dual' of \(G\) associated with an \(H\)-admissible lattice \(\Gamma\subset G\) which converges to the Plancherel measure of \(H\backslash G\) as \(\Gamma\) shrinks to the identity \(\{e\}\) in some way?'', see [\textit{M. Tsuzuki}, J. Funct. Anal. 255, No. 5, 1139--1190 (2008; Zbl 1158.43009)], p. 1140. Here \(H\) is a symmetric subgroup of \(G\) such that \(\Gamma\cap H\) is a lattice in \(H\). We now introduce the setting for the main theorem of the paper which can be seen as a solution to the posed problem above. With \(W\) we denote a \(N\)-dimensional vector space over \(\mathbb C\) equipped with a non-degenerate hermitian form \(\langle\cdot, \cdot \rangle\) of signature \((p,q)\). The group \(G\) is defined by \[ G=\{g\in \text{GL}(W);\quad \langle gx,gy\rangle = \langle x,y\rangle \text{ for all } x, y\in W\}. \] Therefore, \(G\cong U(p,q)\) and \(H\) is defined to be isomorphic to \(U(p-1,q)\times U(1)\), where \(U(p,q)\) is the usual unitary group. Moreover, let \(E/\mathbb Q\) be a CM extension field of finite degree with ring of integers \(\mathcal{O}_E\). With \(F=E\cap \mathbb R\) we denote the maximal real field in \(E\). We now define a lattice of \(G\) associated to an ideal of \(\mathcal{O}_E\) and a lattice \(\mathcal{L}\) of \(W\) for this special setting. We assume that \([F:\mathbb Q]\geq 2\) and set \[ \Gamma_{\mathcal{L}} = \{g\in G;\quad g\mathcal{L} = \mathcal{L}\}, \] and for any \(\mathcal{O}_E\)-ideal \(\mathcal{I}\) \[ \Gamma_{\mathcal{L}}(\mathcal{I}) = \{\gamma\in \Gamma_{\mathcal{L}};\quad \gamma v - v\in \mathcal{I}\mathcal{L} \text{ for all } v\in \mathcal{L}\}. \] Note that \(\mathcal{L}\) and \(\mathcal{I}\) can be chosen such that \(\Gamma_{\mathcal{L}}(\mathcal{I})\) is \(H\)-admissible. Further, we define \(\delta(\mathcal{I})\) to be the minimal norm of elements \(0\not=\lambda\in \mathcal{I}\), where \(\mathcal{I}\) is interpreted as a subset of the Euclidean space \(E\otimes_{\mathbb Q}\mathbb R\). In order to define an appropriate measure on the \(H\)-spherical unitary dual of \(G\) the author makes use of the explicitly known \(H\)-spherical irreducible unitary representations of \(G\) on \(L^2(H\backslash G)\). These are all given in section 7.2.1 of the paper and are not recalled here. They are parameterized by a subset \(S^H = S_{\text{ct}}^H\cup S_{\text{disc}}^H\) of the complex numbers \(\mathbb C\), where \(S_{\text{ct}}^H = \sqrt{-1}\mathbb R^+\cup (0,\nu_0)\) for some \(\nu_o\in \mathbb R^+\) and \(S_{\text{disc}}^H = \{\sigma_d=p-q-1+2d;\quad d\in \mathbb N, \; \sigma_d > 0 \}\). Finally, for a \(H\)-admissible lattice \(\Gamma\subset G\) an appropriate measure \(d\mu_\Gamma^H\) on \(S^H\) is defined in the following way \[ d\mu_{\Gamma}^H = \sum_{\nu\in S^H(\Gamma)}\frac{\|\mathcal{P}_{\Gamma}^H(\Pi_\nu)/\ell^0_{\pi_{\nu}}\|^2}{\text{vol}(\Gamma\cap H\backslash H}\,\delta_\nu, \] where \(S^H(\Gamma) = \{\nu\in S^H;\quad \|\mathcal{P}_{\Gamma}^H(\Pi_\nu)/\ell^0_{\pi_{\nu}}\|^2\not=0\}\), \(\pi_\nu\) is any of the above mentioned \(H\) spherical irreducible representation of \(G\), and \(\delta_{\nu}\) is the Dirac measure. For the definition of \(\mathcal{P}_{\Gamma}^H(\Pi_\nu)/\ell^0_{\pi_{\nu}}\) see p. 1040 of the paper. A Plancherel measure on \(S^H\) is denoted by \(d\mu^H\) and defined on p. 1077 of the paper. The paper ends with the proof of the following main theorem. \textbf{Theorem 2.1.} Let \(\mathcal{L} = \bigoplus_{i=1}^N\mathcal{O}_Eu_i\) be an \(\mathcal{O}_E\)-lattice generated by a \(\mathbb C\)-basis \(\{u_i\}\) of \(W\) such that \(\ell=\mathbb C u_1\). Let \(\{\mathcal{I}_n\}_{n\in\mathbb N}\) be a sequence of \(\mathcal{O}_E\)-ideals such that \(\mathcal{I}_{n+1}\subset \mathcal{I}_n\) for any \(n\in \mathbb N\) and \(\lim_{n\rightarrow \infty}\delta(\mathcal{I}_n) = \infty\). Suppose \(\Gamma_{\mathcal{L}}(\mathcal{I}_0)\) is torsion free, and set \(\Gamma_n = \Gamma_{\mathcal{L}}(\mathcal{I}_n)\) for \(n\in \mathbb N\). Suppose that the condition \(\spadesuit(\tau_d)\) of Theorem 11 (see Theorem 11, p. 1048 for \(\spadesuit(\tau_d)\) and section 2.2 and p. 1075 of the paper for the definition of \(\tau_d\)) is satisfied for \(d\in \mathbb N\) such that \(\sigma_d = 1\) if \(p+q-1\) is odd. Then, \[ \lim_{n\rightarrow \infty}\int_{S^H}f(\nu)\,d\mu_{\Gamma_n}^H = \int_{S^H}f(\nu)\,d\mu^H \] for any \(f\in \mathcal{D}(S^H)\) (see p. 1070 of the paper for definition of \(\mathcal{D}(S^H)\)). In my opinion the paper is quite technical and in this sense not easy to read. Nevertheless, it is thoroughly and precisely written.