On towers of liftings and hypercuspidality for unitary groups (Q685330)

Let \(\pi\) be a cuspidal automorphic representation of the quasi-split unitary group \(U(2,3)\). This paper studies the cuspidality and hypercuspidality properties of the automorphic forms in the space of theta lifts \(\theta^ i (\pi)\) of \(\pi\) to the groups \(U(i,i)\), \(i=1\) to 5. (The theta lifts depend upon the embeddings of \(U(2,3) \times U(i,i)\) into the metaplectic group.) The author first studies cuspidality in the tower. He shows that the lift \(\theta^ i (\pi)\) is cuspidal if and only if \(\theta^{i-1} (\pi)\) is zero. Moreover, \(\theta^ 5 (\pi)\) is always nonzero. These results are analogous to the results of \textit{S. Rallis} concerning towers of theta lifts in the split case [Compos. Math. 51, 333-399 (1984; Zbl 0624.22011)]. Also, by a result of \textit{S. Gelbart}, \textit{D. Rogawski} and \textit{D. Soudry} [Isr. J. Math. 83, 213-252 (1993; Zbl 0789.11033)], the theta lift \(\theta^ i\) is irreducible if it is cuspidal. The author next studies the Whittaker functions (Fourier coefficients) of the theta lifts. He expresses the Whittaker coefficient of the lift of \(\pi\) to \(U(3,3)\) in terms of the Whittaker function for \(\pi\), and thereby shows that the lift \(\theta^ 3\) is already nonzero for generic \(\pi\). Accordingly, the lifts \(\theta^ 4\) and \(\theta^ 5\) can be cuspidal only if \(\pi\) is hypercuspidal, i.e. all Whittaker functions are zero. In that case, the author shows that these lifts themselves are hypercuspidal. Analogous results for the lower rank tower \(U(1,2)\) to \(U(i,i)\), \(i=1\) to 3, were established by \textit{T. Watanabe} [Tôhoku Math. J., II. Ser. 39, 259-279 (1987; Zbl 0634.10025)].