Distinguished representations for unitary groups (Q1365145)

This paper is concerned with a unitary group \(G=U(2r,2r)\) of a quadratic extension \(K/k\) of number fields, and a symplectic subgroup \(H=Sp(r,r,D)\) associated with a quaternion algebra \(D\). It is observed that Fourier type local orbital integrals \(\int f(u^{-1}\xi\theta(u))\psi(u)du\) for some characters \(\psi\not=1\) of a unipotent subgroup \(U\) of \(G\) are nonzero for a single orbit of \(\xi\), and such nonzero integrals are related to similar integrals on the Siegel type Levi subgroup \(G'\) of \(G\). This is an analogue of considerations in the case where both \(K/k\) and \(D\) split [\textit{H. Jacquet} and \textit{S. Rallis}, Pac. J. Math. 154, 265-283 (1992; Zbl 0778.11032)], where it was noted that automorphic forms on \(GL(2n)\) with \(Sp(n)\)-periods are not cuspidal but residual, parametrized by forms on \(GL(n)\). The publication of the present paper might be justified if it serves as a technical step towards a determination of cusp forms on \(G\) with \(H\)-periods. Reviewer's remark: The introduction states without reference that ``it is known that the Asai \(L\)-function \(L_{\text{Asai}} (s,\sigma)\) has a pole if \(\sigma\) is distinguished with respect to \(GL(n,k)\)''; a reference is: \textit{Y. Flicker}, Bull. Soc. Math. Fr. 116, 295-313 (1988; Zbl 0674.10026). Further, the introduction mis-states the precise conjecture of \textit{Y. Flicker} [J. Reine Angew. Math. 418, 139-172 (1991; Zbl 0725.11026)] as asserting that distinguished representations of \(GL(n,K)\) with respect to \(GL(n,k)\) arise as unstable base change lifts from unitary groups (the conjecture is: unstable if \(n\) is even, but stable if \(n\) is odd). This conjecture also appeared in the reviewer's May 1990 NSA proposal, reviewed by an author of \textit{H. Jacquet} and \textit{Y. Ye} [C. R. Acad. Sci., Paris, Sér. I 311, 671-676 (1990; Zbl 0715.11026)]; the latter work is an analogous conjecture in a dual case (representations of \(GL(n,K)\) with periods by a unitary group). Previously \textit{Y. Ye} [J. Reine Angew. Math. 400, 57-121 (1989; Zbl 0665.10020)] stated that certain representations of \(GL(2,K)\) with \(GL(2,k)\)-periods come from \(GL(2,k)\), not from a unitary group.