On the linearity problem for mapping class groups (Q5942400)

A longstanding question is whether mapping class groups of surfaces can be faithfully represented into matrix groups, that is, whether they are linear. Recently \textit{S. Bigelow} [Braid groups are linear, J. Am. Math. Soc. 14, No. 2, 471-486 (2001; Zbl 0988.20021)] proved that braid groups are linear (see also \textit{D. Krammer} [Turkish J. Math. 142, 451-486 (2000; Zbl 0988.20023)]), and \textit{M. Korkmaz} [On the linearity of certain mapping class groups, Turkish J. Math. 24, No. 4, 367-371 (2000; Zbl 0965.57013)] has shown that the mapping class group of the closed surface of genus \(2\) is linear. These results are surprising in that the automorphism groups of free groups \(\text{Aut}(F_n)\), which often exhibit characteristics similar to those of mapping class groups, were shown to be nonlinear for \(n\geq 3\) by \textit{E. Formanek} and \textit{C. Procesi} [J. Algebra 149, No. 2, 494-499 (1992; Zbl 0780.20023)]. The method of Formanek and Procesi is to imbed certain nonlinear groups into the \(\text{Aut}(F_n)\). In the paper under review, the authors prove that no group of the type used by Formanek and Procesi can imbed in any surface mapping class group, and thereby remove a threat to their possible linearity. NEWLINENEWLINENEWLINEAmong the excluded nonlinear subgroups are the HNN extensions \(H(G)=(G\times G)*_{\mathbb Z}\), where the HNN generator acts by \(t(g,g)t^{-1}=(1,g)\). The entire family of excluded groups consists of the \(H(G)/K\) where \(K\) is a normal subgroup such that the image of \(G\times\{1\}\) in \(H(G)/K\) is not nilpotent-by-abelian-by-finite. These are called FP groups by the authors. The main result is that no surface mapping class group (including those of punctured and/or bounded surfaces, relative to the boundary) contain FP groups. The proof uses \textit{N. V. Ivanov}'s concept of pure mapping classes [Subgroups of Teichmüller modular groups, Translated from the Russian by E. J. F. Primrose and revised by the author, Trans. Math. Monogr. 115 (1992; Zbl 0776.57001)]. These are classes that (up to isotopy) fix a finite set of loops in the surface, and on each complementary piece are either the identity or pseudoAnosov. Any mapping class group has a finite index subgroup that consists of pure classes. The centralizers of pure mapping classes can be effectively analyzed, and it is the properties of these centralizers that the authors find to be inconsistent with the existence of an FP subgroup. NEWLINENEWLINENEWLINEThe authors also include a proof of a weaker result, of interest because it is based on completely different methods. Formanek and Procesi detected the presence of FP groups using the fact that if \(H\) is any group and \(\text{Aut}(H)\) contains a free group \(F_2\) that acts on \(H\) in a certain way, then \(\text{Aut}(H)\) contains an FP group. The free groups with this particular action on \(H\) are called poison subgroups. The weaker result is that the mapping class group of a boundaryless surface with exactly one puncture contains no poison subgroup. The method involves showing that such a poison subgroup would lead to a certain decomposition of the fundamental group of the surface as the fundamental group of a graph of groups, then arguing that the action of the poison subgroup on the fundamental group would be inconsistent with this structure.