On the transfer of traces of classical \(p\)-adic groups to a twisted linear group (Q995990)

Let \(F\) be a non-Archimedean local field and let \(n\) be a non-negative integer. Let \(\theta\) denote the automorphism of \(\text{GL}(n,F)\) defined by \(\theta(g)=J{}^tg^{-1}J\) for \(g\in \text{GL}(n,F)\), where \(J\) is the antidiagonal matrix \[ J=\begin{pmatrix} 0&\cdots&0&0&1\\ 0&\cdots&0&-1&0\\ \cdots&\cdots&&\cdots&\\ 0&(-1)^{n-2}&0&\cdots&0\\ (-1)^{n-1}&0&0&\cdots&0\end{pmatrix}. \] Let \(\rho\) be a unitary irreducible supercuspidal representation of a group \(\text{GL}(d_\rho,F)\) (which defines the integer \(d_\rho\)). For any integer \(a\), let \(\rho\nu^{(a-1)/2} \times\cdots\times\rho\nu^{-(a-1)/2}\), with \(\nu=|\det|_F\), denote the parabolically induced representation from \(\text{GL}(d_\rho,F)^a\) to \(\text{GL}(ad_\rho,F)\) of the representation \(\rho\nu^{(a-1)/2} \otimes\cdots\otimes\rho\nu^{-(a-1)/2}\). Let \(\text{St}(a,\rho)\) denote the unique irreducible submodule of \(\rho\nu^{(a-1)/2} \times\cdots\times\rho\nu^{-(a-1)/2}\). Then for any integer \(b\), let \(\text{Sp}(b,\text{St}(a,\rho))\) denote the unique irreducible quotient of the parabolically induced representation \[ \text{St}(\rho,a)\nu^{(b-1)/2} \times\cdots\times\text{St}(\rho,a)\nu^{-(b-1)/2}, \] from \(\text{GL}(ad_\rho,F)^b\) to the group \(\text{GL}(abd_\rho,F)\) of the representation \[ \text{St}(\rho,a)\nu^{(b-1)/2} \otimes\cdots\otimes\text{St}(\rho,a)\nu^{-(b-1)/2}. \] Now let \[ \sigma:=\times_{(\rho,a,b)}\,\text{Sp}(b,\text{St}(a,\rho)), \] where \((\rho,a,b)\) runs over the set of triples, such \(\rho\) is a unitary irreducible supercuspidal representation of a group \(\text{GL}(d_\rho,F)\), and \(a\) and \(b\) are integers with \(\sum_{(\rho,a,b)} abd_\rho=n\). Let \(\tilde{G}(n)=\text{GL}(n,F)\rtimes\langle 1,\theta\rangle\). If all the representations \(\rho\) which occur in the above product are such that their isomorphy class is invariant under the action of \(\theta\) (such representations are called \(\theta\)-invariant), then the representation \(\sigma\) of \(\text{GL}(n,F)\) is also \(\theta\)-invariant, and hence can be extended to a representation (still denoted by \(\sigma\)) of the group \(\tilde{G}(n)\). This depends on a choice. The possible choices are studied in the paper under review as well as the signs which occur when a change of choice is made. One of the main purposes of the paper is to reduce the computation of the trace \(\text{tr}(\sigma(g,\theta))\) for any \(g\in\text{GL}(n,F)\) to the computation of the trace for tempered representations. The representation \(\sigma\) defined above is reinterpreted by the way of morphisms \[ \psi\colon W_F\times\text{SL}(2,\mathbb C)\times\text{SL}(2,\mathbb C)\to\text{GL}(n,\mathbb C) \] such that the restriction of \(\psi\) to \(W_F\) is continuous and bounded. We write \(\sigma=\pi(\psi)\). Let \(\Delta\) denote the diagonal embedding of \(\text{SL}(2,\mathbb C)\) into \(\text{SL}(2,\mathbb C)\times\text{SL}(2,\mathbb C)\). The tempered representations which occur are those which are attached to the morphism \(\psi\circ\Delta\) for all the Levi subgroups of \(\text{GL}(n,F)\) such that the dual group contains a conjugate of \(\psi\circ\Delta\). By using the Jacquet functor, the authors reduce the study of the general case to the case where \(\psi\circ\Delta\) is \textit{discrete} (that is, when it defines a semisimple representation of \(W_F\times\text{SL}(2,\mathbb C)\) without multiplicity). The difficult case is when \(\psi\circ\Delta\) is discrete. In that case, the centralizer of \(\psi\) and those of \(\psi\circ\Delta\) are both products of groups \(\text{GL}(1,\mathbb C)\) and the centralizer of \(\psi\) is contained in the centralizer of \(\psi\circ\Delta\). In the case where they are equal (that is, when \(\psi\), as a representation of \(W_F\times\text{SL}(2,\mathbb C)\times\text{SL}(2,\mathbb C)\), is a sum of irreducible representations such that each of them is trivial on at least one copy of \(\text{SL}(2,\mathbb C)\)), the morphism \(\psi\) is said to be \textit{elementary}. The authors deal with the case when \(\psi\) is elementary by realizing the representation \(\pi(\psi)\) as the cohomology of a complex coming from the tempered representation defined by \(\psi\circ\Delta\) and which is equipped by an action of \(\theta\). The reduction of the case where \(\psi\circ\Delta\) is discrete to the special case where \(\psi\) is elementary is done by a combinatoric work where the representation attached to \(\psi\) is realized in the Grothendieck group with basis induced representations attached to elementary morphisms.