The automorphism group of the hyperelliptic Torelli group (Q2360797)

For a closed orientable surface \(S_g\) of genus \(g\), the \textit{hyperelliptic Torelli group} is the intersection of the hyperelliptic mapping class group (the centralizer in the mapping class group of a fixed hyperelliptic involution \(\iota\), unique up to conjugacy) and the Torelli group (the subgroup acting trivially on the homology of the surface). The main result of the present paper then states that, for \(g \geq 3\), the automorphism group of the hyperelliptic Torelli group is isomorphic to the factor group \(\text{SMod}^\pm(S_g)/\langle \iota \rangle\) of the extended hyperelliptic mapping class group \(\text{SMod}^\pm(S_g)\) (allowing also orientation-reversing mapping classes now) by the subgroup generated by the hyperelliptic involution (then it is also isomorphic to the extended mapping class group of a sphere with \(2g+2\) punctures). This is inspired by analogous results of several authors for the mapping class group \(\text{Mod}(S_g)\) itself as well as for the Torelli group whose automorphism groups are both isomorphic to the extended mapping class group \(\text{Mod}^\pm(S_g)\). ``The key step in each of their arguments was to find an appropriate complex for the group and consider the automorphism group of that complex. Motivated by this, we define an abstract simplicial flag complex, the \textit{symmetric seperating curve complex}, whose 1-skeleton is given by vertices corresponding to isotopy classes of symmetric separating curves, and edges are between vertices with disjoint representatives. A simple closed curve is \textit{symmetric} if it is fixed by the hyperelliptic involution.'' The second main result of the paper states that the automorphism group of the symmetric seperating curve complex is again isomorphic to the factor group \(\text{SMod}^\pm(S_g)/\langle \iota \rangle\), in analogy with a result of Ivanov that the automorphism group of the modular group itself is isomorphic to the automorphism group of the classical curve complex.