On injective homomorphisms between Teichmüller modular groups. I (Q1283486)

For a compact connected orientable surface \(S\), the Teichmüller modular group \(\text{Mod}_S\) of \(S\), also called the mapping class group of \(S\), is the group of isotopy classes of orientation-preserving diffeomorphisms \(S\to S\). This group plays a central role in low-dimensional topology, but its algebraic structure is not well understood. One of the problems about the algebraic structure of \(\text{Mod}_S\) is to understand endomorphisms \(\text{Mod}_S\), or more generally homomorphisms between two mapping class groups. This paper proves some results in this direction. A group \(G\) is called residually finite if for each \(x\neq 1\) in \(G\) there exists a homomorphism \(f\) from \(G\) onto some finite group \(Q\) such that \(f(x)\neq 1\) in \(Q\). \(G\) is called Hopfian if every surjective endomorphism of \(G\) is an automorphism. It can be shown that finitely generated residually finite groups are Hopfian. The dual notion is the co-Hopfian property: A group \(G\) is called co-Hopfian if every injective endomorphism of \(G\) must necessarily be an automorphism of \(G\). It is well known that the mapping class group of an orientable surface is finitely generated and residually finite, thus it is Hopfian. The paper under review proves that it is also co-Hopfian if the genus of the surface is at least \(2\). This is a special case of the main result of the paper: Let \(S\) and \(S'\) be compact connected orientable surfaces. Suppose that the genus of \(S\) is at least \(2\) and \(S'\) is not a closed surface of genus \(2\). If the maxima of the ranks of abelian subgroups of \(\text{Mod}_S\) and \(\text{Mod}_{S'}\) differ by at most one and if \(\rho : \text{Mod}_S \to \text{Mod}_{S'}\) is an injective homomorphism, then there is a diffeomorphism \(H:\text{Mod}_S\to \text{Mod}_{S'}\) such that \(\rho ([G])=[HGH^{-1}]\) for every orientation-preserving diffeomorphism \(G:S\to S\). In particular, \(\rho\) is an isomorphism. Another application of the main result is the answer to a question of Birman: If \(S\) is a closed orientable surface of genus at least \(2\), then there is no injective homomorphism \(\text{Out}(\pi_1(S)) \to \text{Aut}(\pi_1(S))\). In particular, the natural epimorphism \(\text{Aut}(\pi_1(S)) \to \text{Out}(\pi_1(S))\) is nonsplit. The corresponding results for the case where \(S\) has genus \(1\) is also stated in the paper and is promised to be proved in a sequel to the present paper.