The Laplacian for domains in hyperbolic space and limit sets of Kleinian groups (Q1087671)

Let \(\Omega\) be an open connected subset of the \((n+1)\)-dimensional hyperbolic space \({\mathbb{H}}^{n+1}\) and define the quadratic forms \[ H(f,g)=\int_{\Omega}f\bar gdV\quad (f,g\in L^ 2(\Omega)), \] \(D(u,v) = \int_{\Omega}<\nabla u,\overline{\nabla v}>dV\) \((u,v\in W^ 1(\Omega))\). Consider the (positive) self-adjoint Laplacian (for the hyperbolic metric) \(\Delta\) with Neumann boundary conditions. The domain of \(\Delta\) is the set of all \(u\in W^ 1(\Omega)\) such that \(\Delta u\in L^ 2(\Omega)\) and \(H(\Delta u,v)=D(u,v)\) for all \(v\in W^ 1(\Omega)\). Let \(\sigma(\Omega)\) denote the spectrum of \(\Delta\). The main aim of the paper under review is a careful study of the dependence of \(\sigma(\Omega)\) on \(\Omega\). Section 1 gives a detailed introduction and the main results. Section 2 is devoted to the discussion of various quadratic forms such as H, D and the energy form \(E(u)=D(u,u)-(n/2)^ 2H(u,u)\). Note that E need not be positive definite. The authors construct an auxiliary form K such that K is compact with respect to \(G:=E+K\). This yields that there is a closed subspace of \({\mathbb{H}}_ G\) \((=\) completion of \(W^ 1(\Omega)\) w.r.t. G) of finite codimension on which E is positive. Using this fact, the authors prove the following. Let \({\mathbb{F}}_ m\) be the set of domains \(\Omega\) which are convex and bounded by exactly m geodesic hyperplanes. If \(\Omega \in {\mathbb{F}}_ m\) and \(vol(\Omega)=\infty\), then \(\sigma(\Omega)\) is continuous in \([(n/2)^ 2,\infty [\) and discrete in \([0,(n/2)^ 2[\). The spectrum of \(\Delta\) in \([0,(n/2)^ 2[\) is non- empty if and only if E takes on negative values. - It is possible for \(\Delta -(n/2)^ 2\) on \({\mathbb{H}}_ G\) to have 0 in its discrete spectrum. The corresponding eigenfunction v is called a null vector and satisfies \(E(v)=0\). (Note that \(v\not\in W^ 1(\Omega).)\Omega\) is called free if \(E\geq 0\), and if moreover \(\Delta -(n/2)^ 2\) has no null vector, then \(\Omega\) is called strictly free. Free and strictly free domains play a crucial rôle as basic building blocks in this paper, and there are several interesting results related with these concepts. Section 3 is concerned with obtaining lower bounds for \[ \lambda_ 0(\Omega):=\inf \{D(u): u\in W^ 1(\Omega),\quad H(u)=1\}. \] By way of example we quote: If \(\Omega\) is bounded by a k-coplanar set of hyperplanes, then \(\lambda_ 0(\Omega)\geq k(n-k)\), and \(\Omega\) is free if moreover \(k=[n/2]\) (Theorem 3.2). If \(\Omega \in {\mathbb{F}}_ m\) has at most \([n/2]+2\) sides, then \(\Omega\) is free (Theorem 3.7). The latter result is modified in section 5 to the effect that regardless of the number of sides, \(\Omega\) will be free if its sides are sufficiently separated. - Adapting standard terminology from the theory of Kleinian groups, the set \(\Omega\) is called a Schottky domain if \(\Omega\) is bounded by non-intersecting hyperplanes. For each \(n\geq 3\) there is a constant \(d_ n>0\) such that for any Schottky domain \(\Omega\) in \({\mathbb{H}}^{n+1}\), \(\lambda_ 0(\Omega)\geq d_ n\) (Theorem 3.9). There are several related results that cannot be incorporated in this review. In Section 4, the authors prove continuity results for the dependence of \(\sigma\) (\(\Omega)\) on \(\Omega\) under the assumption that \(\Omega\) be geometrically finite. In particular, conditions for the validity of \(\lambda_ 0(\Omega_ k)\to \lambda_ 0(\Omega)\) for \(\Omega_ k\to \Omega\) are obtained (Theorem 4.4). The investigations of the work under review are mainly motivated by the spectral theory of the Laplacian for Kleinian groups acting on \({\mathbb{H}}^{n+1}\). In Section 5 the authors consider the special case of a domain \(\Omega\) which is a fundamental domain of a Kleinian group \(\Gamma\) acting on \({\mathbb{H}}^{n+1}\). In this case the Laplacian also defines a self-adjoint operator on \(L^ 2(\Gamma \setminus {\mathbb{H}}^{n+1})\), and we denote by \(\lambda_ 0(\Gamma)\leq \lambda_ 1(\Gamma)\leq \lambda_ 2(\Gamma)\leq..\). the discrete spectrum (if it exists; otherwise \(\lambda_ 0(\Gamma)\) denotes the corresponding infinum), by \(d(\Lambda_{\Gamma})\) the Hausdorff dimension of the limit set \(\Lambda_{\Gamma}\) of \(\Gamma\) and by \(\delta(\Gamma)\) the abscissa of convergence of the Poincaré series for \(\Gamma\). Then the Patterson- Sullivan theorem states: (i) If \(\delta(\Gamma) \geq n/2\), then \(\lambda_ 0(\Gamma)=\delta (\Gamma)(n-\delta(\Gamma))\). (ii) If \(\Gamma\) is geometrically finite, then \(\delta (\Gamma)=d(\Lambda_{\Gamma})\). - It is easy to see that \(\lambda_ j(\Omega) \leq \lambda_ j(\Gamma)\) with equality holding if \(\Gamma\) is a reflection group and \(\Omega\) is a reflection domain. For every \(n\geq 3\) there exists a \(c_ n<n\) such that \(\delta (\Gamma)\leq c_ n\) for any Schottky group \(\Gamma\) ; if \(\Gamma\) is geometrically finite, then also \(d(\Lambda_{\Gamma})\leq c_ n\) (Theorem 5.4). It turns out that null vectors exist in abundance: If \(\Omega\) is a free Schottky domain without cusps, then \(\Omega\) has a null vector if and only if \(\delta (\Gamma)=n/2\) (Theorem 5.7). The results of the present work are supplemented by various examples and numerical calculations of the Hausdorff dimensions of the limit sets of many Schottky groups (see Section 6). In particular, the Hecke group \(\Gamma_{\mu}\) (generated by \(z\to z+\mu\) with fixed \(\mu >2\) and \(z\mapsto -1/z)\) has precisely one discrete eigenvalue \(\lambda_ 0(\mu)\). As \(\mu\) ranges from 2 to \(\infty\), \(\lambda_ 0(\mu)\) increases from 0 to 1/4 (Theorem 6.1). The number \(\lambda_ 0(\mu)\) was investigated more closely by the authors [Duke Math. J. 52, 211-221 (1985; Zbl 0564.30030)]. Since it is impossible to summarize the contents of this long paper in a nutshell, the reader desiring full information is advised to read the authors' detailed introduction and the paper: \textit{P. Sarnak} [Singularities and dynamical systems, Proc. Int. Conf., Heraklion/Greece 1983, North-Holland Math. Stud. 103, 485-500 (1985; Zbl 0578.30038)].