On the Tannaka group attached to the theta divisor of a generic principally polarized abelian variety (Q897495)

Principally polarised abelian varieties (ppavs) have been studied intensively for a long time but there are still many open questions. The one addressed here concerns a group \(G(\delta_\Theta)\), called the Tannaka group, attached to a ppav with a choice of an irreducible theta divisor \(\Theta\). In practice \(\Theta\) is chosen to be symmetric: the Tannaka group is then well-defined, at least for generic abelian varieties. The group \(G(\delta_\Theta)\) may be thought of as being the algebraic group whose category of representations is equivalent to the category of subquotients of convolution powers of \(\delta_\Theta\oplus \delta_\Theta^\vee\), where the convolution product \(K_1*K_2\) in the derived category is \(Ra_*(K_1\boxtimes K_2)\), induced by addition. Two questions arise immediately. Can one calculate this group at all, in any cases and especially in interesting ones such as Jacobians; and what does the resulting stratification of the moduli space look like? Partial answers to both questions are given here. Specfically, it is shown that for general abelian varieties one has \[ G(\delta_\Theta)\cong {\text{Sp}}(g!,{\mathbb C}) \] for \(g\) even, and \[ G(\delta_\Theta)\cong {\text{SO}}(g!,{\mathbb C}) \] for \(g\) odd. The invariant \(G(\delta_\Theta)\) is semicontinuous (i.e.\ special cases are smaller) and the stratification is finite and constructible. (It is already known what \(G(\delta_\Theta)\) is for Jacobians, and it is smaller -- the exact group depends on whether the curve is hyperelliptic or not.) Particular attention is paid to the case \(g=4\), which is the first nontrivial case. In that case the locus where the group is not \({\text{Sp}}(24,{\mathbb C})\) is the Andreotti-Mayer locus, and the invariant distinguishes the two components, the Jacobian locus and the locus of surfaces with a vanishing theta-null. In Section~2 a simply duality argument is used to bound the \(G(\delta_\Theta)\) from above. Using the constructibility property (proved in an appendix, Section~7) one can then obtain the lower bounds by a highly non-trivial degeneration argument. The description of \(G(\delta_\Theta)\) given above makes sense for an arbitrary perverse sheaf, and the groups thus obtained from the perverse sheaves of nearby cycles on the central fibre are contained in the Tannaka group of the general fibre of a degeneration. The deformation theory needed for this is carried out here (Section~3) in more generality than is needed (in mixed characteristic), with a view to future applications. These perverse sheaves thus induce (Section~4) large irreducible representations of (parts of) \(G(\delta_\Theta)\), and in the case of degenerations to Jacobians, the theta null locus or products we get enough information to reduce the problem one that can be solved by standard techniques in representation theory (Section~5). The remaining Section~6 presents an alternative argument that avoids the need to use the theta null degenerations, which is perhaps more universally applicable.