The isomorphism problem for hyperbolic groups. I (Q1842632)

In 1911 Dehn formulated three fundamental problems for groups which are the word, the conjugacy and the isomorphism problem (and solved them for the canonical presentations of the fundamental groups of closed surfaces). All three problems are unsolvable, in general, so one has to restrict to reasonable subclasses of groups; however, in contrast to the word problem, the isomorphism problem could be solved only for some rather isolated classes of groups so far. In the present paper, the isomorphism problem is solved for the class of torsion-free hyperbolic groups (as introduced and popularized by Gromov as a reasonable large but accessible class of groups), under the additional hypothesis that the hyperbolic group does not admit an essential small action on a real tree. This hypothesis is equivalent to the fact that the outer automorphism group of the hyperbolic group is finite. (A second part of the paper is announced giving a solution of the isomorphism problem for general torsion-free hyperbolic groups). The presented algorithm gives also an effective computation of the (finite) outer automorphism group of such groups, and permits to decide if the outer automorphism group of a general torsion-free hyperbolic group is finite. In combination with some other results, the algorithm implies solutions of the homeomorphism problem for closed hyperbolic manifolds, for closed negatively curved manifolds of dimension \(\geq 5\) and for ``rigid weakly geometric'' 3-manifolds, as well as solutions of the conjugacy problem for pseudo-Anosov automorphisms of surfaces and for irreducible automorphisms of free groups. The main tools of the algorithm include Makanin's theorem on the decidability of the Diophantine theory of free semigroups with paired alphabet, canonical representatives which are special kinds of quasi-geodesics introduced and used in a paper by Rips and the author to solve equations over hyperbolic groups, and some canonical forms of monomorphic images of balls in the Cayley graph of one hyperbolic group in the Cayley graph of the other. The whole procedure is directed toward revealing all possible conjugacy classes of monomorphisms between the two groups in question (which is a finite set under the above hypothesis on the groups): ``Since one is not able to impose the monomorphic conditions on a homomorphism globally, we divide our procedure into a sequence of consecutive steps where at each step we require homomorphisms between the two groups to be monomorphic on some ball in the Cayley graph of the first group.'' The main steps of the general approach are described in more detail in the first section of the paper.