Periods of residual automorphic forms (Q2338940)

A representation of a reductive connected group \(G\) is said to be distinguished by a subgroup \(H\) if it admits a nonzero \(H\)-invariant linear form. Distinguished representations are expected to be those representations of \(G\) which are obtained as functorial lift from a third group \(G'\). This concept makes sense for representations of real or \(p\)-adic groups as well as automorphic representations where the requirement on a nonzero invariant linear form is replaced by the nonvanishing of the \(H\)-period \(\int_{H(F)\backslash H(\mathbb A)}\varphi(h)\,dh\), when \(\varphi\) is a cusp form. This paper deals with a generalization to noncuspidal automorphic forms and representations having no Whittaker model. When \(\varphi\) is not cuspidal, the period integral will be divergent. However, one can still construct a natural regularization of the divergent \(H\)-period integral, called regularized period, in various cases. The regularization is based on a relative variant of Arthur's truncation operator, called mixed truncation and denoted by \(\Lambda^T_m\). The mixed truncation enjoys many good properties. Its upshot is a rapidly decreasing function on \(H(F)\backslash H(\mathbb A)^1\). This paper applies this machinery to the case of the twisted tensor, or Asai, \(L\)-function, where \(G = R_{E/F}\mathrm{GL}(n)\) and \(H = \mathrm{GL}(n)/F\), and \(E/F\) is a quadratic extension of number fields. The distinction in this case has been extensively studied both locally and globally. In this case the group \(G'\) is conjectured to be a quasisplit unitary group of \(n\) variables by the reviewer's work: [J. Reine Angew. Math. 418, 139--172 (1991; Zbl 0725.11026)]. In particular the paper shows: Let \(\sigma\) be an irreducible cuspidal automorphic representation of \(\mathrm{GL}(r,A_E)\). Then the residual representation \(\mathrm{Sp}(\sigma,d)\) is distinguished by \(\mathrm{GL}(dr,\mathbb A)\) if and only if \(\sigma\) is distinguished by \(\mathrm{GL}(r,\mathbb A)\).