Fourier transform and the global Gan-Gross-Prasad conjecture for unitary groups (Q483228)

Informally speaking, the global Gan-Gross-Prasad conjectures for unitary groups state that a cuspidal automorphic representation of a product \(\mathrm{U}_{n+1}\times \mathrm{U}_n\), of unitary groups (in \(n+1\) and \(n\) variables) has a \(\Delta \mathrm{U}_n\)-distinguished member in its near equivalence class (conjecturally a single Vogan \(L\)-packet) if and only if the Rankin-Selberg \(L\)-function of its weak base change does not vanish at \(1/2\). The main result of this paper proves this conjecture for representations that satisfy certain simplifying hypotheses. More precisely: Theorem 1. Let \(E/F\) be a quadratic extension of number fields, and \(W\hookrightarrow V\) an embedding of nondegenerate \(E/F\)-Hermitian spaces whose dimensions are \(n\) and \(n+1\). Let \(\pi\) be a cuspidal automorphic representation of \(\mathrm{U}(V)\times \mathrm{U}(W)\), which has a weak base change \(\Pi= \Pi_{n+1}\times\Pi_n\) to the group \(G_{n+1}'\times G_n',\) where \(G_m'= \text{Res}_{E/F}\text{GL}_m\) for all \(m\). Assume that hypothesis \((*)\) and conditions (1) and (2) (see further below) hold. Then the following are equivalent: (i) \(L(1/2, \Pi_{n+1}\times\Pi_n)\neq 0\) (nonvanishing of a central \(L\)-value); and (ii) There exists an embedding \(W'\hookrightarrow V'\) of \(E/F\)- Hermitian spaces of dimensions \(n\) and \(n+1\), such that some automorphic representation \(\pi'\) of \(\mathrm{U}(V')\times \mathrm{U}(W')\) that is nearly equivalent to \(\pi\) is distinguished by the subgroup \(\Delta \mathrm{U}(W')\subset \mathrm{U}(V')\times \mathrm{U}(W')\). Here, the condition of being distinguished in (ii) means that there exists an element \(\phi_{\pi^\prime}\) in the space of \(\pi'\) such that: \[ 0\neq \int_{\Delta \mathrm{U}(W')(F)\setminus\Delta \mathrm{U}(W')(\mathbb{A}_f)}\phi_\pi(h)\,dh.\tag{1} \] To say that \(\pi\) and \(\pi'\) are nearly equivalent means that \(\pi_v\cong\pi_v'\) for almost all places \(v\) (which makes sense, as for almost all places \(v\) an isomorphism \(V_v\cong V_v'\) exists and may be harmlessly fixed). Here: \(\bullet\) Hypothesis \((*)\) refers to the existence of weak base change for unitary groups, which has been proved by \textit{C. P. Mok} [Mem. Am. Math. Soc. 1108, iii-v, 248 p. (2015; Zbl 1316.22018)] for quasi-split groups and Kaletha-Minguez-Shin-White for their inner forms. \(\bullet\) Condition 1 says that every Archimedean place is split in \(E/F\) (needed because `smooth transfer' is proved only for a non-Archimedean local field in this paper); and \(\bullet\) Condition 2 says that \(\pi_{v_1}\), \(\pi_{v_2}\) are supercuspidal for two distinct non-Archimedean \(v_1\), \(v_2\) split in \(E/F\) -- this is so as to be able to work with nice enough `test functions' which allow one to get away without a fine spectral expansion of the Jacquet-Rallis relative trace formula. The local (tempered or generic) version of this conjecture is known without any restrictive hypothesis, thanks to the work of R. \textit{R. Beuzart-Plessis} [Compos. Math. 151, No. 7, 1309--1371 (2015; Zbl 1328.22013); ``La conjecture locale de Gross-Prasad pour les représentations tempérées des groupes unitaires'', Preprint, \url{arXiv:1205.2987}] (who followed the work of \textit{J.-L. Waldspurger} [in: Sur les conjectures de Gross et Prasad. II. Paris: Société Mathématique de France (SMF). 103--165 (2012; Zbl 1276.22010)] and \textit{C. Mœglin} and \textit{J.-L. Waldspurger} [in: Sur les conjectures de Gross et Prasad. II. Paris: Société Mathématique de France (SMF). 167--216 (2012; Zbl 1276.22007)] for orthogonal groups). Various small rank global cases and some partial results, too numerous to summarize here, had also been known. Recently, the author has used this approach to prove, again under some simplifying restrictions, a more precise relation between the left side of equation (1) and the central \(L\)-value \(L(1/2,\Pi_1\times\Pi_2)\) of interest (this is \textit{R. N. Harris}' refinement [Int. Math. Res. Not. 2014, No. 2, 303--389 (2014; Zbl 1322.11047)], following the work of \textit{A. Ichino} and \textit{T. Ikeda} [Geom. Funct. Anal. 19, No. 5, 1378--1425 (2010; Zbl 1216.11057)] on orthogonal groups, of the global Gan-Gross-Prasad conjecture). As an application the author proves as Theorem 1.2 the following statement: Theorem 2. If a cuspidal automorphic representation \(\sigma\) of \(\text{GL}_{n+1}(\mathbb{A}_E)\) is a weak base change of a cuspidal automorphic representation \(\pi\) of a unitary group \(\mathrm{U}(V)\) that is locally supercuspidal at two split places, then \(L(1/2,\sigma\times\tau)\neq 0\) for some cuspidal automorphic representation \(\tau\) of \(\text{GL}_n(\mathbb{A}_E\)). Note that this follows from Theorem 1 if one proves that there is a cuspidal automorphic representation \(\pi_2\) of some \(\mathrm{U}(W)\), \(W\hookrightarrow V\) with codimension 1, such that \(\pi_1\otimes\pi_2\) is distinguished by \(\Delta \mathrm{U}(W)\) (for one can then simply take \(\tau\) to be a weak base change of \(\pi_2\), whose existence is guaranteed by the hypothesis of interest). The existence of such \(W\) and \(\pi_2\) (under the hypotheses at hand) is given by a ``Burger-Sarnak principle type'' variant of a result of D. Prasad. To avoid being technical, we will be rather imprecise in what follows. We will only crudely list some of the main ideas the proof of Theorem 1: \(\bullet\) A key step is to prove an equality -- a `relative trace identity' -- of the form: \[ I_\Pi(f')= \sum_W\, \sum_{\pi_W} J_{\pi_W}(f_W),\tag{2} \] where -- \(\Pi\) is a cuspidal automorphic representation of \(G':= \text{Res}_{E/F}(\text{GL}_{n+1}\times \text{GL}_n)\); -- \((W,V)\) runs over a suitable set of pairs of Hermitian spaces (suitably related so as to be determined by \(W\)); -- \(\pi_W\) runs over a suitable subset of a near equivalence class of automorphic representations of \(G_W:= \mathrm{U}(V)\times \mathrm{U}(W)\), -- \(I_\Pi\), `global spherical character', is a distribution whose nonvanishing forces \(\Pi\) to be a weak base change from some \(G_W\) and also to have nonvanishing \(L(1/2,\Pi_{n+1}\times\Pi_n)\); -- \(J_{\pi_W}\), again a global spherical character, is a distribution whose nonvanishing is equivalent to \(\pi_W\) being distinguished with respect to \(\Delta \mathrm{U}(W)\); -- \(f'\in C^\infty_c(G'(\mathbb{A}))\) and the tuple \((f_W)_W\), \(f_W\in C^\infty_c(G_W(\mathbb{A}))\) are ``nice'' functions related to each other by what is called `smooth transfer' (`nice' entails that at some finite place they are required to be `essentially a supercuspidal matrix coefficient', and at some other, not necessarily finite, place, supported on a suitably regular semisimple locus). \(\bullet\) Let us give an idea of why equation (2) gives (ii)\,\(\Rightarrow\)\,(i). Because of what has been said above about \(I_\Pi\), it is enough to get \(I_\Pi(f')\neq 0\) for a suitable \(f'\). If \(\pi_W\) is distinguished, \(J_{\pi_W}\) is nonzero. To apply equation (2), however, one needs to choose a nice \(f_W\) so that \(J_{\pi_W}(f_W)\neq 0\), and also show that there exists an \(f'\) as in that equation. The existence of \(f_W\) uses the factorization of the global spherical character \(J_{\pi_W}\) into local ones \(J_{\pi_{W,v}}\), each of which is `of positive type', and also a technical input, proved in Appendix A by A. Ichino and the author, regarding the support of \(J_{\pi_{W,v}}\), which is necessary to ensure that \(f_W\) can indeed be chosen to be nice. One also needs to show that `smooth transfer' exists at the non-Archimedean places (part of the reason for the restrictive hypotheses in Theorem 1 is the nonavailability of this result at the Archimedean primes). The arguments need the relevant fundamental lemma (for the Jacquet-Rallis relative trace formula), which was proved in positive characteristic by \textit{Z. Yun} [Duke Math. J. 156, No. 2, 167--227 (2011; Zbl 1211.14039)] and transferred to characteristic zero by J. Gordon. \(\bullet\) As one key input in proving the remaining implication, namely (i)\,\(\Rightarrow\)\,(ii), one needs to prove one direction of the Flicker-Rallis conjecture under some assumptions -- namely, that weak base changes to \(\text{Res}_{E/F}\text{GL}_n\) from some unitary groups are `Flicker-Rallis' distinguished (distinguished by \(\text{GL}_n(F)\) if \(n\) is odd, and distinguished by \((\text{GL}_n(F),\eta\circ\text{det})\) if \(n\) is even, \(\eta= \eta_{E/F}\) denoting the class field theory character associated to \(E/F\)). Towards proving this result, the author proves a result that specializes to the variant of the result of D. Prasad mentioned earlier. \(\bullet\) The proof of equation (2) is an implementation of a simple form of the Jacquet-Rallis trace formula: -- One expresses either side of the equation in terms of orbital integrals; orbital integrals (twisted by a character) for the action of \(H_1'\times H_2':=\Delta(\text{Res}_{E/F})\text{GL}_n\times (\text{GL}_{n+ 1,F}\times \text{GL}_{n,F})\) on \(G'\) by left and right multiplication for the left-side, and orbital integrals for the action of \(H\times H:=\Delta \mathrm{U}(W)\times\Delta \mathrm{U}(W)\) on \(G\) by left and right multiplication for the right-side. -- The condition of `smooth transfer' means that the orbital integrals on either side match up to yield the desired equality. Here, expressing the left (resp., right) side of equation (2) using orbital integrals is done by interpreting it as an integral over \(H_1'(F)\setminus H_1'(\mathbb{A})\times H_2'(F)\setminus H_2'(\mathbb{A})\) (resp., \(H(F)\setminus H(\mathbb{A})\times H(F)\setminus H(\mathbb{A})\)) of a kernel function seen in the usual Arthur-Selberg trace formula. Note that the \(H_1\) here relates to the Rankin-Selberg period and \(H_2\) to the Flicker-Rallis period. A technical input that goes into getting the exact expression is an automorphic Cebotarev density theorem due to D. Ramakrishnan. \(\bullet\) One then needs to prove the existence of the smooth transfer at \(p\)-adic places. One also proves a partial result in the Archimedean case. In other words, given \(f'\), we want to find an (\(f_W)_W\) with matching orbital integrals, and vice versa. This is an important result of the paper, which will of course continue to be relevant even after the fine spectral expansion is developed. Some of the steps to proving this may be very sketchily summarized as: -- To study orbital integrals for \(H_1'\setminus G'/H_2'\), one can identify \(H_1'\setminus G'\) with \(\text{Res}_{E/F} \text{GL}_{n+1}\). Further, by identifying \(\text{Res}_{E/F} \text{GL}_{n+1,F}\) with the subvariety \(S_{n+1}\) of \(\text{Res}_{E/F} \text{GL}_{n+1}\) defined by \(s\overline s= 1\) (via \(g\mapsto g\overline g^{-1}\)), one is reduced to studying orbital integrals for the conjugation action of \(\text{GL}_n(F)\) on \(S_{n+1}(F)\) (as mentioned above, these orbital integrals are twisted by a character). Similarly, the orbital integrals for \(H\times H\) on \(G= \mathrm{U}(V)\times \mathrm{U}(W)\) reduce to those of conjugation by \(\mathrm{U}(W)\) on \(\mathrm{U}(V)\). -- One can use a Cayley map to reduce these questions to the level of the Lie algebra (this involves, for instance, understanding the behavior of the transfer factors across the Cayley map). -- One then uses semisimple descent to reduce the question to one of transfer around the zero element in certain (Luna) sliced representations. This uses an explicit construction of the relevant Luna slices that the author provides in Appendix B. -- In Section 4, the author proves transfer around zero at the `Lie algebra level', by adapting ideas from Waldspurger's famous work on fundamental lemma implying transfer. Thus, the author develops the local version of the relative trace formula, and studies the behavior of Fourier transform with respect to smooth matching.