Splittings and automorphisms of relatively hyperbolic groups (Q2356014)

Summary: We study automorphisms of a relatively hyperbolic group \(G\). When \(G\) is one-ended, we describe \(\mathrm{Out}(G)\) using a preferred JSJ tree over subgroups that are virtually cyclic or parabolic. In particular, when \(G\) is toral relatively hyperbolic, \(\mathrm{Out}(G)\) is virtually built out of mapping class groups and subgroups of \(\mathrm {GL}_n(\mathbb Z)\) fixing certain basis elements. When more general parabolic groups are allowed, these subgroups of \(\mathrm{GL}_n(\mathbb Z)\) have to be replaced by McCool groups: automorphisms of parabolic groups acting trivially (i.e. by conjugation) on certain subgroups. Given a malnormal quasiconvex subgroup \(P\) of a hyperbolic group \(G\), we view \(G\) as hyperbolic relative to \(P\) and we apply the previous analysis to describe the group \(\mathrm{Out}(P\nearrow G)\) of automorphisms of \(P\) that extend to \(G\): it is virtually a McCool group. If \(\mathrm{Out}(P\nearrow G)\) is infinite, then \(P\) is a vertex group in a splitting of \(G\). If \(P\) is torsion-free, then \(\mathrm{Out}(P\nearrow G)\) is of type VF, in particular finitely presented. { } We also determine when \(\mathrm{Out}(G)\) is infinite, for \(G\) relatively hyperbolic. The interesting case is when \(G\) is infinitely-ended and has torsion. When \(G\) is hyperbolic, we show that \(\mathrm{Out}(G)\) is infinite if and only if \(G\) splits over a maximal virtually cyclic subgroup with infinite center. In general we show that infiniteness of \(\mathrm{Out}(G)\) comes from the existence of a splitting with infinitely many twists, or having a vertex group that is maximal parabolic with infinitely many automorphisms acting trivially on incident edge groups.