Big symplectic or orthogonal monodromy modulo \(\ell\) (Q2465261)

Let \(k\) be a field, \(C/k\) a proper smooth geometrically connected curve, \(K=k(C)\) and \(E/K\) an elliptic curve with nonconstant \(j\)-invariant. \textit{E. Kowalski} [Int. J. Number Theory 2, No. 2, 267--288 (2006; Zbl 1115.11030)] considered if one could prove uniform big \({\mathbb F}_ {\ell}\)--monodromy results for certain sequences of quadratic twists of \(E\) when \(k={\mathbb F} _ {\ell}\) and showed how a sufficiently strong uniform bound as \(\ell\) varies, allows one to prove a variant of Goldfeld's average-rank conjecture. The goal of this paper is to complete Kowalski's proof by providing bounds of a sort that he requires. Also, the author shows that his methods can be used to give a new proof of a result of Yu which had given evidence of the existence of sufficient bounds required by Kowalski. The main result states that if \(V\) is an \({\mathbb F}_ {\ell}\)-vector space with a perfect pairing \(\varphi: V\times V\to {\mathbb F}_ {\ell}\), \(G\leq \roman{GL}(V)\) is an irreducible primitive subgroup preserving \(\varphi\). If \(\varphi\) is symmetric, \(G\) contains a reflection and an isotropic shear, and if \(\ell \geq 5\), then \(G\) is either: \({\mathcal O}(V)\), the orthogonal group; the kernel of the spinor norm; or the kernel of the product of the spinor norm and the determinant. If \(\varphi\) is alternating, \(G\) contains a transvection, and \(\ell\geq 3\), then \(G\) is all of the symplectic group \(\text{Sp}(V)\). The result of Yu proved is that the \(\bmod \ell\) geometric monodromy of hyperelliptic curves is \(\text{Sp}(2g,{\mathbb F}_ {\ell})\) for \(\ell>2\). In the last section, the author gives applications of the group theorem to families of quadratic twists of elliptic curves, and derives new results for the inverse Galois problem showing how big subgroups of orthogonal groups arise as Galois groups over \({\mathbb Q}(t)\).