Maximal subgroups of the modular and other groups (Q667024)

Let \(\mathcal{N}\) be a \textit{Neumann map of type \((p,q)\)}, where \(p,q\) are integers with \(p\geq3,\;q\geq2\). This is an infinite bipartite graph, embeddable in an oriented surface, the valencies of whose vertices are assumed to be \(1\), \(p\) or \(q\). It is also assumed that \(\mathcal{N}\) has a single \textit{face}. Let \(\Omega\) denote the (countably infinite) set of the edges of \(\mathcal{N}\) (which are directed for the case \(q=2\)). In a previous important paper the author and \textit{D. Singerman} [Proc. Lond. Math. Soc. (3) 37, 273--307 (1978; Zbl 0391.05024)] describe the action of a subgroup \(G=\langle x,y\rangle\) of \(S^{\Omega}\), where \(x^p=y^q=1\). Identifying \(\Omega\) with \(\mathbb{Z}\), the action of \(z=(xy)^{-1}\) on \(\mathbb{Z}\) can be assumed to be the translation \(t \mapsto t+1\). Let \(\Gamma\) denote the triangle group \[ \Delta(p,q,\infty)=\langle X,Y,Z\mid X^p=Y^q=XYZ=1\rangle. \] Then \(\Gamma\) maps (naturally) onto \(G\). Let \(M\) be the lift of any stabilizer in \(G\). Then \[ \Gamma=MP \text{ and } M \cap P=1, \] where \(P=\langle Z\rangle\). A subgroup with these properties is called a \textit{Neumann subgroup} of \(\Gamma\). It follows that \(M \cap P^{\Gamma} =\{1\}\) and that \(M\) is maximal with respect to this property. Extending existing terminology from the modular group, \(\mathrm{PSL}_2(\mathbb{Z})\) (\( \cong \Delta(3,2,\infty)\)), a Neumann subgroup is said to be a \textit{maximal nonparabolic subgroup} of \(\Gamma\). The subgroup \(M\) is in addition a maximal subgroup of \(\Gamma\) if and only if the action of \(G\) is primitive. In this case, \(M\) is called a \textit{nonparabolic maximal subgroup} of \(G\). The principal result in this paper is the following. Theorem. For each pair \(p\geq 3,\;q\geq 2\) the group \(\Gamma=\Delta(p,q,\infty)\) has uncountably many conjugacy classes of nonparabolic maximal subgroups. This impressive paper sheds new light on many existing results. \textit{B. Neumann} [Sitzungsber. Preuß. Akad. Wiss., Phys.-Math. Kl. 1933, 429--444 (1933; JFM 59.0146.03)] has earlier proved that the modular group \(\Delta(3,2,\infty)\) contains uncountably many maximal nonparabolic subgroups. Neumann's method however is based on a very complicated construction of permutations of \(\mathbb{Z}\). The author is able to show that these subgroups of Neumann are a special case arising from the theory of Neumann maps which provides a much more transparent proof. Moreover, the author is able to use this approach to determine precisely which of Neumann's examples are actually maximal subgroups. Any subgroup of \(\Gamma\cong C_p \star C_q\) is by the Kurosh subgroup theorem a free product of subgroups \(C_d\), where \(d\) divides \(p\), \(q\) or \(C_{\infty}\). It follows that if the subgroup is generated by elements of finite order, which applies to the original subgroups of Neumann, there are no free factors in its decomposition. \textit{C. Tretkoff} [Glasg. Math. J. 16, 91--102 (1975; Zbl 0321.20033)] has proved that the modular group has Neumann subgroups whose decomposition does involve a free factor and consequently do not arise from Neumann's original construction. The author indicates how Tretkoff's examples [loc. cit.] can be derived from Neumann maps with more that one face. \textit{J. L. Brenner} and \textit{R. C. Lyndon} [Math. Ann. 263, 1--11 (1983; Zbl 0491.20037)] have shown that the modular group has maximal nonparabolic subgroups which are not Neumann subgroups. Using the theory of Neumann maps with more than one face, the author indicates (in detail for the modular group) that every \(\Gamma\) has uncountably many conjugacy classes of nonparabolic maximal subgroups which are not Neumann subgroups.