Whittaker-orthogonal models, functoriality, and the Rankin-Selberg method (Q1204255)

The purpose of this paper is to exhibit a phenomenon in the Rankin- Selberg method, in which a Rankin-Selberg integral of an automorphic form on one group \((SO(2n+2))\) is used to represent an \(L\)-function of an automorphic form on a different group \((Sp(2n)\times GL(m))\). The two automorphic forms involved are related by a lifting of the sort described by the Langlands' functoriality conjecture. The role played by Whittaker models in the Rankin-Selberg method is taken over by a modification of it which they call a Whittaker-Orthogonal model (WO-model). In section 2, they prove that an irreducible admissible representation of \(GO(2n+2,F)^ 0\) for a non-archimedean local field \(F\) admits up to constant multiple at most one WO-functional. In section 3, they show that a nonramified principal series representation of this group has a WO- model if and only if it is a local lifting from \(GSp(2n)\), in which case they give an explicit formula for the spherical vector in the model analogous to the Casselman-Shalika formula for the spherical vector in a Whittaker model. In section 4, they give analogous results for the group \(SO(2n+2)\). From these results they represent for a nonramified place \(v\) the local factor \[ L_ v((m+2)s-(m+1)/2, \quad \pi_ v'\otimes \tau_ v, \text{ ``standard''}) \] of an \(L\)-function on \(Sp(2n)\times GL(m)\) with \(n\geq 2\) and \(m\leq n-1\) by the corresponding local factor of the Rankin-Selberg integral \[ \int_{X_ F\setminus X_ A} \int_{W_ F\setminus W_ A} \int_{SO_{2m+3}(F)\setminus SO_{2m+3}(A)} \varphi(xwg)\psi(w) E(g,s,\tau,f_ s) dg dw dx \] on \(SO(2n+2)\) by regarding \(SO(2m+3)\) as a subgroup of \(SO(2n+2)\). They also discuss the case \(m\geq n\).