Separation of series for hyperboloids (Q1128366)

By \textit{I. M. Gelfand} and \textit{S. G. Gindikin} [Funkt. Anal. Appl. 11, 258-265 (1978); translation from Funkts. Anal. Prilozh. 11, 19-27 (1978; Zbl 0449.22018)] a program for developing harmonic analysis on homogeneous spaces \(G/H\) was proposed (the so-called Gelfand-Gindikin program). The first step of this program is to decompose \(L^2(G/H)\) into subspaces \(L_j\) on each of which irreducible unitary representations act that belong to a single series. The subsequent analysis must provide a natural description of functions in \(L_j\) (for example, as boundary values of functions holomorphic on complex manifolds), etc. In this paper, the problem of the separation of series for real hyperboloids \(X=SO_0 (p,q)/\) \(SO_0 (p,q-1)\) is solved. As is known, the quasiregular representation on the hyperboloid can be decomposed into two series of irreducible unitary representations, namely, the continuous and discrete series. Therefore, it suffices to indicate the projection onto the subspace of \(L^2(X)\) on which one of these series acts. We give explicit formulas for the projection on the subspace \(L^2_d(X)\) on which the discrete series acts. For the case \(q=2\), the hyperboloid \(X\) is the Shilov boundary of some complex manifolds, and the discrete series splits into two series, namely, the analytic and antianalytic series. In [J. Funct. Anal. 147, 26-50 (1997; Zbl 0881.43005)] we obtained the projections related to each of these series, together with the corresponding Cauchy-Szegö kernels. For completeness we write out the formulas for these projections in the present paper. Note that in the general case (for arbitrary \(q)\), the discrete series is related to a certain analyticity, see formulas (1.1) and (3.4), (3.3) for the projection and for the spherical functions. The problem on the separation of series for hyperboloids was considered by \textit{S. G. Gindikin} [J. Geom. Phys. 10, 175-184 (1993; Zbl 0787.22014)], where, in another form the projection related to the discrete series in the case of odd \(n\) was obtained \((n=p+q)\). As can be seen from the present paper (see also [the author, op. cit.]), in this circle of problems, the case in which \(n\) is odd is simpler than that in which \(n\) is even; the method of \textit{S. G. Gindikin} [op. cit.] fits for odd \(n\) only.