Central limit theorem and infinitely divisible probabilities associated with partial differential operators (Q1900158)

The spherical functions of the Gelfand pair \((U(n,1), U(n))\) are given in terms of two-dimensional Jacobi functions \(\varphi^\alpha_{\lambda, \mu}\) with \(\alpha = \alpha (n)\); this was shown in a series of classical papers of Flensted-Jensen and Koornwinder. Motivated by the associated double coset hypergroups \(U(n,1)//U(n)\), the author studies two- dimensional commutative hypergroup structures whose characters are two- dimensional Jacobi functions \(\varphi^\alpha_{\lambda, \mu}\) with \(\alpha \in \mathbb{R}\), \(\alpha \geq 0\). The main purpose is to present some results in algebraic probability theory for these and related hypergroups. Unfortunately, the associated convolutions are not defined properly for some cases which leads to some incorrect statements about these convolutions. To describe this, consider the underlying spaces and the associated convolutions. By Theorem 3.1 of \textit{M. Flensted-Jensen} [Math. Ann. 228, 65-92 (1977; Zbl 0331.43013)], there is a covering group \(\widetilde G\) of \(U(n,1)\) containing \(U(n)\) in a natural way such that \(\widetilde G//U(n)\) can be regarded as a commutative hypergroup on \([0, \infty [\times \mathbb{R}\) with a convolution given via product formulas for two- dimensional Jacobi functions. It was observed by \textit{K. Trimeche} [J. Math. Pures Appl., IX. Sér. 70, 1-73 (1991; Zbl 0671.35004)] that these formulas hold for all \(\alpha \in \mathbb{R}\), \(\alpha \geq 0\). They also lead to commutative hypergroups on \(K : = [0, \infty [\times \mathbb{R}\) such that \(\{0\} \times \mathbb{R}\) is a subgroup isomorphic to \((\mathbb{R},+)\). In this way, for each \(\lambda > 0\), the set \(G_\lambda : = \{0\} \times \lambda \mathbb{Z}\) is a subgroup of \(K\), and the quotient \(K/G_\lambda\) carries a unique commutative quotient hypergroup structure isomorphic with an appropriate hypergroup on \([0, \infty [\times {\mathbf T}\) or \(\{z \in \mathbb{C} : |z |\geq 1\}\). The author claims to construct convolutions on \([0, \infty [\times [- l \pi, l \pi]\) \((l \in \mathbb{N})\) and \(K\). For this, he defines the convolution on \(\{z \in \mathbb{C} : |z |\geq 1\}\) (the same as above for \(\lambda = 1)\); however, he then omits the transfer to \([0, \infty [\times [- l \pi, l \pi]\) and \(K\). This transfer is clear for \(l = 1\) only, and in all other cases the convolution must be defined carefully to obtain consistent associative and continuous convolutions. Moreover, from a formal point of view, one has to take the torus \({\mathbf T}\) instead of \([- l \pi, l \pi]\). Due to these gaps, some statements in Sections 1 and 2 are obscured. In particular, parts of Theorem 2.1 are incorrect. Sections 3 and 4 of the paper are devoted to probability theory on the above hypergroups. The main result of Section 3 is a triangular central limit theorem (with strong assumptions) where the limit distribution is normal. Finally, Section 4 deals with convolution semigroups, infinitely divisible measures, the Lévy-Khinchin representation, as well with the transience. The nontrivial proof of the Lévy-Khinchin formula is very consise. On the other hand, the transience criterion can be directly deduced from growth properties of the underlying hypergroup [cf. Theorem 6.3.16 in the recent monograph of \textit{W. R. Bloom} and \textit{H. Heyer}, ``Harmonic analysis of probability measures on hypergroups'' (1995; Zbl 0828.43005)].