The most continuous part of the Plancherel decomposition for a reductive symmetric space (Q1355870)

Let \(G/H\) be a semisimple symmetric space; that is, \(G\) is a connected semisimple real Lie group with involution \(\sigma\), and \(H\) is an open subgroup of the group of fixed points for \(\sigma\) in \(G\). A fundamental problem of harmonic analysis on \(G/H\) is to obtain an explicit direct integral (Plancherel) decomposition of \(L\simeq \int^\oplus_{\widehat G} m_\pi\pi d\mu (\pi)\) of the regular representation \(L\) of \(G\) in \(L^2 (G/H)\) into irreducible unitary representations. The principal result of this paper is the determination of such a decomposition for the part of \(L^2(G/H)\) which corresponds to the ``most continuous part'' of the spectrum. Let \({\mathfrak g}\) be the Lie algebra of \(G\) and let \({\mathfrak g}= {\mathfrak h}+ {\mathfrak q}\) be its decomposition into \(\pm 1\)-eigenspaces for \(\sigma\) (so that \({\mathfrak h}\) is the Lie algebra of \(H)\). Furthermore, let \(\theta\) be a Cartan involution commuting with \(\sigma\). By analogy with the group case one expects \(L^2(G/H)\) to decompose into a finite number of parts, each attached to a particular parabolic subgroup \(P\) satisfying \(\sigma\theta (P)=P\). For a \(\sigma\)-minimal parabolic subgroup \(P=MAN\) let \(\xi\) be a finite dimensional unitary representation of \(M\) and let \(\lambda\in {\mathfrak a}^*_{q\mathbb{C}}\), where \({\mathfrak a}_q ={\mathfrak a} \cap {\mathfrak q}\). Furthermore, let \(C^\infty (\xi: \lambda)\) and \(C^{-\infty} (\xi: \lambda)\) denote the spaces of smooth, respectively generalized, vectors for the representation \(\pi_{\xi, \lambda}\) associated to \(\sigma\) and \(\lambda\), and let \(C^\infty (\xi: \lambda)^H\) be the space of \(H\)-fixed elements in \(C^{-\infty} (\xi: \lambda)\). In [\textit{E. P. van den Ban}, Ann. Sci. Éc. Norm. Supér, IV. Sér. 21, 359-412 (1988; Zbl 0714.22009)] and [\textit{E. P. van den Ban} and \textit{H. Schlichtkrull}, Fourier transforms on a semisimple symmetric space (to appear in Invent. Math.)] a certain finite dimensional space \(V(\xi)\) (independent of \(\lambda)\) was defined together with a natural family of linear maps \(j^0(\xi: \lambda) =j^0 (P:\xi: \lambda): V(\xi)\to C^{-\infty} (\xi: \lambda)^H\), depending meromorphically on \(\lambda\in {\mathfrak a}^*_{q\mathbb{C}}\) and bijective for generic \(\lambda\). The definition involved a normalization procedure which guarantees that \(\lambda \mapsto j^0 (\xi: \lambda)\) is regular at the imaginary points \(\lambda\in i {\mathfrak a}^*_q\). The Fourier transform \(\widehat f\) for \(f\in C^\infty_c (G/H)\) is then defined by \[ \widehat f(\xi: \lambda)= \int_{G/H} f(gH) \pi_{\xi,- \lambda} (g)j^0 (\xi: \lambda) d(gH)\in \Hom \bigl(V(\xi), C^\infty(\xi: -\lambda) \bigr). \] The main result of the paper asserts that if one sets \(d\mu (\xi,\lambda) =\dim(\xi)d \lambda\), where \(d\lambda\) is the suitably normalized Lebesgue measure on \(i{\mathfrak a}^*_q\), then \(f\mapsto \widehat f\) extends to a partial isometry of \(L^2(G/H)\) onto the space \(\int^\oplus_{\xi, \lambda} m_\xi \pi_{\xi, \lambda} d\mu (\xi,\lambda)\) which specified domains for \(\xi\) and \(\lambda\). The proof of these results uses uniform estimates for the coefficients of a converging expansion for the Eisenstein integral, estimates of Paley-Wiener type for the Fourier transform \({\mathcal F}f\) of a Schwartz function \(f\), the wave packet transform \({\mathcal I} \varphi\) of a \(^0{\mathcal C} (\tau)\)-valued function \(\varphi\) on \(i{\mathfrak a}^*\) on \(G/H\) and other methods. In the last section the authors derive a Paley-Wiener type result for spaces of split rank one. The image by \({\mathcal F}\) of the space of compactly supported smooth functions of type \(\tau\) is characterized by a growth condition.