An invariant trace formula for rank one lattices (Q4704686)

As a far reaching generalization of the Selberg trace formula, \textit{J. Arthur} [J. Am. Math. Soc. 1, 323-383 (1988; Zbl 0682.10021)] proved an adelic invariant trace formula for lattices \(\Gamma\) of semisimple groups \(G\). The aim of this paper is to prove such an invariant formula for rank one lattices of real semisimple Lie groups. There are common features between both trace formulae, but unlike the toy case of the modular group [e.g. \textit{M. Duflo} and \textit{J.-P. Labesse}, Ann. Sci. Éc. Norm. Supér., IV. Sér. 4, 193-284 (1971; Zbl 0277.12011)], there are differences in the scope of the results: Arthur's trace formula applies in any rank, where rank one is worked out only by Hoffmann; geometric terms of Hoffmann's trace formula are made explicit while analysis remains to be done for some terms in Arthur's formula. The methods are clearly different, however the general philosophy is common. In an earlier work [J. Funct. Anal. 84, 373-440 (1989; Zbl 0699.22016)], the author proved a trace formula for lattices. Unlike for the trace formula for uniform lattice, not all geometric terms are invariant distributions: these new terms correspond spectrally to the presence of the continuous spectrum, geometrically to the cusps of the manifold or the parabolic subgroups \(P\) fixed by some unipotents from \(\Gamma\). The main achievement of the current paper is the expression of the geometric side of the trace formula as a sum of terms, all of which are invariant functions on the group \(G\). The proof proceeds by geometric and spectral analysis. The first sections concern the geometry of parabolic subgroups and orbital integrals induced by unipotents, following the work by \textit{M. S. Osborne} and \textit{G. Warner} [J. Reine Angew. Math. 324, 1-113 (1981; Zbl 0469.22007)]. Then Eisenstein series, constant term and scattering matrix are thoroughly studied, through a reduction to the spectral analysis on the reductive pair \((M,\Gamma\cap M)\), with \(M\) a component in the Levi decomposition of the parabolic \(P\) where the analysis of uniform lattice (because of the rank one assumption) is available. This invariant trace formula should be applied elsewhere to analyse the general Selberg zeta function associated to natural bundles on non-compact locally symmetric spaces, extending known results for compact bases [see \textit{D. Fried}, Ann. Sci. Éc. Norm. Supér., IV. Sér. 19, 491-517 (1986; Zbl 0609.58033)] or trivial bundles [as by \textit{R. Gangolli} and \textit{G. Warner}, J. Math. Soc. Japan 27, 328-343 (1975; Zbl 0325.22014)].