Some results on Archimedean Rankin-Selberg integrals (Q2346659)

The author only works with smooth finitely generated admissible Fréchet representation of \(G_n := {\mathrm{GL}}_n({\mathbb{R}})\) of moderate growth in the sense of Casselman and Wallach. Let \(N_n\) denotes the subgroup of unipotent upper triangular matrices and let \(\psi : N_n \rightarrow {\mathbb{C}}^\times\) be a non-degenerate character. A standard induced representation of \(G_n\) from a discrete series representation of a parabolic subgroup is called a representation in general position if the continuous parameter is non-integral and non-half integral (see p. 284 in the paper for a precise definition). A representation \((\pi, V_\pi)\) in general position is irreducible and generic, i.e., \(V_\pi\) processes a nonzero continuous Whittaker functional \(\Lambda\) with respect to \(\psi\). Then the function \(W_v : g \mapsto \Lambda(\pi(g)v)\) for \(g \in G_n\) is called a Whittaker function of \(V_\pi\). The space \({\mathcal{W}}(\pi,\psi)\) of all called Whittaker functionals of \(V_\pi\) is called the Whittaker model of \(\pi\). In [\textit{I. N. Bernshteĭn} and \textit{A. V. Zelevinskiĭ}, Russ. Math. Surv. 31, No. 3, 1--68 (1976; Zbl 0348.43007)], the \(l\)-th derivative of an algebraic representation of \({\mathrm{GL}}_n(F)\) was introduced where \(F\) is a \(p\)-adic field. There is an analogous Archimedean \(l\)-th derivative of the representation \((\pi,V_\pi)\) of \(G_n\) which we will denote by \((\pi^{(l)}, V_\pi^{(l)})\). It is a smooth representation of \(G_{n-l}\). Let \(\pi\) and \(\pi'\) be irreducible generic representations of \(G_n\) and \(G_m\) respectively. In this review, we will only explain the case \(n = m\). Let \({\mathcal{S}}_k\) denote the space of Schwartz functions on \({\mathbb{R}}^k\). We define \[ I_k(s,W,W',\Phi) = \int_{N_k \backslash G_k} W \begin{pmatrix} g & 0 \\ 0 & I_{n-k} \end{pmatrix} W' \begin{pmatrix} g & 0 \\ 0 & I_{n-k} \end{pmatrix} \Phi(\epsilon_k g) |\det g|^{s-n+k} dg \] where \(W \in {\mathcal{W}}(\pi,\psi), W' \in {\mathcal{W}}(\pi',\psi^{-1}), \epsilon_n = (0,\ldots,0,1) \in {\mathbb{R}}^n\), \(s \in {\mathbb{C}}\) and \(\Phi \in {\mathcal{S}}_k\). We call \(I_n\) a Rankin-Selberg integral for \(\pi\) and \(\pi'\). It is known that the integral converges for if \(\Re(s)\) is large, has meromorphic continuation to \({\mathbb{C}}\) and satisfies a functional equation. Suppose \[ I_k(s,W,W',\Phi) = \dfrac{B_{s_0,k}(W,W',\phi)}{(s-s_0)^d} + \ldots \] is the Laurent expansion with a pole of order \(d\) at \(s_0\). We say that \(s_0\) is an exceptional pole of type 1 with level \(m\) and depth \(n-k\) if \(B_{s,k}\) is nonzero on \({\mathcal{S}}_k^m\) but \(B_{s,k}|_{{\mathcal{S}}_k^{m+1}} \equiv 0\). Here, \({\mathcal{S}}_k^m\) is the subspace of \({\mathcal{S}}_k\) which vanishes to order at least \(m\) at 0. Note that \(B_{s_0,k}\) is a trilinear form on \(V \times V' \times ({\mathcal{S}}^m_k/{\mathcal{S}}^{m+1}_k)\). If there is a continuous trilinear form \(l\) on \(V_\pi \times V_{\pi'} \times ({\mathcal{S}}^m_n/{\mathcal{S}}^{m+1}_n)\) satisfying \[ l(gW; gW', g v) = |\det g|^{-s_0} l(W,W',v) \] for all \(g \in G_n\), then we say that \(s_0\) is an exceptional pole of type 2 with level \(m\). A type 1 pole with level \(m\) and depth 0 is a type 2 pole with level \(m\). We can now state the main result of the paper: Given two irreducible generic representations \(\pi\) and \(\pi'\) of \(G_m\) in general position. If \(s_0\) is a pole of the Rankin-Selberg integral \(I_n\) for \(\pi\) and \(\pi'\), then there is a \(0 \leq k \leq n-1\) such that \(s_0\) is an exceptional pole of type 2 for some irreducible component of \(\pi^{(k)}\) and \(\pi'{}^{(k)}\). Conversely, an exceptional pole of type 1 of depth \(0\) for some irreducible components of the derivatives \(\pi^{(k)}\) and \(\pi'{}^{(k)}\) is also a pole of \(I_n\). There is also a theorem for the case when \(\pi'\) is a smooth irreducible representation of \(G_m\) in general position where \(m < n\). We refer to Theorem 6.5 in the paper for the precise statement. It is expected that the poles of Rankin-Selberg \(I_n\) are exactly the set of exceptional poles of type 1 for components of derivatives of \(\pi\) and \(\pi'\). The author states in the introduction that this question will be addressed in the future.