Generalized Bernstein-Reznikov integrals (Q621302)

The main result gives a closed formula for a triple integral in the following form \[ \begin{multlined} {\int_{S^{2n-1}\times S^{2n-1}\times S^{2n-1}}\, |[Y,Z]|^{\alpha-n\over 2} |[Z,X]|^{\beta-n\over 2} |[X,Y]|^{\gamma-n\over 2} d\sigma(X)d\sigma(Y)d\sigma(Z)}\\ = (2\pi^{n-{1\over 2}})^3 {\Gamma({2-n+\alpha\over 4})\Gamma({2-n+\beta\over 4})\Gamma({2-n+\gamma\over 4})\Gamma({\delta+n\over 4}) \over \Gamma(n)\Gamma({n-\lambda_1\over 2})\Gamma({n-\lambda_2\over 2})\Gamma({n-\lambda_3\over 2})}\end{multlined} \] with \(d\sigma\) the Euclidean measure on the sphere \(\text{S}^{2n-1}\) in \({\mathbb R}^{2n}\), \[ \alpha=\lambda_1-\lambda_2-\lambda_3,\,\beta=-\lambda_1+\lambda_2-\lambda_3,\,\gamma= -\lambda_1-\lambda_2+\lambda_3,\,\delta=-\lambda_1-\lambda_2-\lambda_3(=\alpha+\beta+\gamma) \] and the \textit{symplectic form} on \({\mathbb R}^{2n}={\mathbb R}^n\oplus {\mathbb R}^n\) is given by \[ [(x,\xi),(y,\eta)]:=-\langle x,\eta\rangle +\langle y,\xi\rangle, \] with \(\langle\;,\;\rangle\) the canonical inner product on \({\mathbb R}^n\). The intergal converges absolutely if and only if \((\lambda_1,\lambda_2,\lambda_3)\in{\mathbb C}\) lies in the non-empty region defined by \[ \begin{align*}{&\text{Re}\,\alpha>n-2,\,\text{Re}\,\beta>n-2,\,\text{Re}\,\gamma>n-2\ \qquad\qquad\quad (n\geq 2),\cr &\text{Re}\,\alpha>n-2,\,\text{Re}\,\beta>n-2,\,\text{Re}\,\gamma>n-2,\,\text{Re}\,\delta>-1\ (n=1).}\end{align*} \] The case \(n=1\) was previously established by \textit{J. Bernstein} and \textit{A. Reznikov} [Mosc. Math. J. 4, No. 1, 19--37 (2004; Zbl 1081.11037)]. The proof is given by interpreting the triple product integral as the trace of a certain integral operator, using the Fourier transform, the classical Bochner identity and reduction to a special value of a hypergeometric \({}_5F_1\) function; the proof is new even in the case \(n=1\). \vskip0.2cm The layout of the paper is as follows: \S1 Triple product integra formula, \S2 Eigenvalues of the integral transform (intermediate results for \S3), \S3 Proof of Theorem 1.1, \S4 Other triple integral formulas (using \(|X-Y|\) and \(|\langle x,y\rangle |\) instead of the symplectic form \([X,Y]\)), \S5 Perspectives from representation theory, \S6 Convergence of the triple integral.