Salem numbers, spectral radii and growth rates of hyperbolic Coxeter groups (Q6097861)

A Coxeter system \((W, S)\) is a pair with \(W\) a group and \(S\) a set of generators that are reflections and the only relations arise from the angles of reflecting hyperplanes. A Coxeter group admits a faithful representation where the generators act as reflections on \(\mathbb{R}^N\). A product of all the generators of \(W\) is called a Coxeter element of \(W\), and the corresponding element in \(\mathrm{GL}_N(\mathbb{R})\) is called a Coxeter transformation. For a Coxeter group, the growth series is defined as \[ f_S(t) = 1 + \sum_{k\ge 1}a_kt^k \] where \(a_k\) is the number of elements of \(S\)-length \(k\) and it is known to be a rational function that depends of the finite subgroups of \(W\). The inverse \(\tau = 1/R = \limsup_{k\to \infty}\sqrt[n]{a_k} \) of the radius of convergence of the series is called the growth rate of \(W\). A Coxeter polyhedron \(P \subset \mathbb{H}^n\) is a convex polyhedron where all the dihedral angles are \({\pi}/m\) where \(m \in \{2, \dots \infty\}\). It is assumed that the polyhedron is of finite volume and thus it is bounded by finitely many hyperplanes. For \(n = 2\), \(P = (p_1, \dots p_k)\) is a hyperbolic polygon where \(p_1, \dots p_k \ge 2\) are integers and \(1/p_1 + \dots + 1/p_k < k - 2\). Reflections on \(\mathbb{H}^n\) with respect to the hyperplanes define a Coxeter group which is called a hyperbolic Coxeter group. It acts on \(\mathbb{H}^n\) with hyperbolic isometries. If \(n\) is 2 or 3, the growth rate is either a quadratic unit or a Salem number, that is a real algebraic integer \(\tau > 1\), whose conjugates have absolute value \(\le 1\) and at least one of them has absolute value 1. The first main result of the paper states that not all Salem numbers appear as growth rates of hyperbolic Coxeter groups. For \(n = 2\), it is proved that the smallest growth rate is \(\tau_{[3, 7]}\) which is Lehmer's number \(\alpha_L\) and after that is \(\tau_{[3,8]}\) which is the seventh smallest number in the known list of Salem numbers. So the numbers in between in the list can not be realised as growth rates of hyperbolic Coxeter groups. For \(n = 3\), it is known that the smallest growth rate is the Salem number \(\tau_{[3,4,3]}\). So the first 47 known numbers are not realised. For \(n \ge 4\), it is proved that the growth rate is either larger that \(\tau_{[3,8]}\) or it is not a Salem number. There is no example of a hyperbolic Coxeter group with \(\tau > \tau_{[3,8]}\). Let \((p_1, \dots p_k)\) be a sequence of integers, \(p_i \ge 2\). We construct the star graph \(\mathrm{Star}(p_1, \dots p_k)\) to be a tree with a vertex with degree \(k\) and \(k\) paths emanating from the vertex of length \(p_i -1\), respectively. Let \((p_1, \dots p_k)\) denote a convex polygon in \(\mathbb{H}^2\) and \(W\) the Coxeter group with Coxeter graph \(\mathrm{Star}(p_1, \dots p_k)\), i.e. with generators corresponding to the vertices of \(\mathrm{Star}(p_1, \dots p_k)\) and relations \((st)^3 = 1\) if two vertices are joined by an edge and \((st)^2 = 1\) otherwise. Then the growth rate of \(W\) equals to the spectral radius of the Coxeter transformation of \(W\). The tree \(H(i,j,k)\) is defined as follows: we start with two vertices \(v_1\) and \(V_2\) of degree 3 and a path of length \(j\) joining them. To \(v_1\) we attach two paths, one having length \(1\) and the other of length \(i - 1\). Similarly, to \(v_2\) we attach a vertex and a path of length \(k-1\). To this graph, we associate a Coxeter group as before. The authors prove that to Coxeter group \([4, 3, 5]\) we associate the graph \(H(2, 8, 3)\) and the Coxeter group \(W\). They prove that \(\tau_{[4, 3, 5]}\) (which is a Salem number) equal to the spectral radius of the Coxeter transformation of \(W\). Also, they prove that \(\tau_{[3,5,3]}\) is not the spectral radius of a Coxeter element of a Coxeter group associated to a graph as above.