Cuspidal integrals for \(\mathrm{SL}(3) / K_\epsilon\) (Q1782258)

In this paper, the authors investigate the notion of cusp forms for some symmetric spaces of split rank 2. Harish-Chandra defined the notion of cusp forms for \textit{reductive Lie groups} and he showed that the space of cusp forms coincides with the closure in the Schwartz space of the span of the discrete series of representations. This fact plays an important role in his work on the Plancherel decomposition for reductive groups. Now it is natural to try to define cusp forms for \textit{reductive symmetric spaces} $G/H$. There is a naive scheme to do it. Following it, one encounters then the problem of convergence of some integrals. Namely, let $\sigma$ denote the involution of $G$ determining the symmetric subgroup $H$. A parabolic subgroup $P$ of $G$ ($P\ne G$) is called $\sigma$-parabolic, if $\sigma(P)$ is opposite to $P$. The integrals are \[ \int_{N_P} \phi (n) \, dn, \] the integration is taken over the normalizers $N_P$ of parabolic subgroups $P$. These integrals have to converge for all $\sigma$-parabolic subgroups $P$ and all functions $\phi$ in the vector space of Schwartz functions $\mathcal{C}(G/H)$. It allows to introduce the Radon transform $\mathcal{R}_P$ by \[ (\mathcal{R}_P \phi) (g) = \int_{N_P} \phi (gn) \, dn, \quad g\in G. \] Then the cusp forms are defined to be those functions $\phi$ for which this transform is equal to zero: $\mathcal{R}_P \phi =0$ for all above-mentioned $P$. For reductive symmetric spaces $G/H$ of \textit{split rank 1}, the problem of convergence of the integrals was solved by \textit{E. P. van den Den Ban} and the second author [Represent. Theory 21, 467--533 (2017; Zbl 1401.22013)]. In the paper under review, the convergence of the integrals in question is investigated for three reductive symmetric spaces of \textit{split rank 2}, namely, (a) $\mathrm{SL}(3,\mathbb{R})/\mathrm{SO}(1, 2)_e$, (b) $\mathrm{SL}(3,\mathbb{C})/\mathrm{SU}(1, 2)$ and (c) $\mathrm{SL}(3,\mathbb{H})/\mathrm{Sp}(1, 2)$. For the spaces (a) and (b) the integrals are absolutely convergent (the main result), while for (c) there exists a Schwartz function such that the integrals are divergent for some of the maximal and all of the minimal parabolic subgroups $P$. Moreover, for the spaces (a) and (b) one can characterize the different series of representation occurring in the Plancherel decomposition of these spaces with the use of these integrals. Namely, let $\mathcal{P}$ and $\mathcal{Q}$ be the sets of all minimal and maximal $\sigma$-parabolic subgroups of $G$, respectively. The space $L^2(G/H)$ decomposes as \[ L^2(G/H) = L^2_{\mathcal{P}}(G/H) + L^2_{\mathcal{Q}}(G/H) \] where $L^2_{\mathcal{P}}(G/H)$ denotes a $G$-invariant closed subspace that is unitarily equivalent to a direct integral of representations that are induced from a parabolic subgroup contained in $\mathcal{P}$, and similarly for $\mathcal{Q}$. Intersecting both sides of the decomposition with $\mathcal{C}(G/H)$ one obtains a decomposition \[ \mathcal{C}(G/H) = \mathcal{C}_{\mathcal{P}}(G/H) + \mathcal{C}_{\mathcal{Q}}(G/H), \] where $\mathcal{C}_{\mathcal{P}}(G/H)= \mathcal{C}(G/H) \cap L^2_{\mathcal{P}}(G/H)$, and similarly for $\mathcal{Q}$. The above-mentioned characterization of series is as follows. Consider two intersections: firstly, the intersection of the kernels of the Radon transforms $\mathcal{R}_P$ in $\mathcal{C}(G/H)$ when $P$ ranges over $\mathcal{P}$ and, secondly, the same replacing $\mathcal{P}$ by $\mathcal{Q}$. Then the first intersection is $\mathcal{C}_{\mathcal{Q}}(G/H)$ and the second one is $\{0\}$.