A tableau formula for vexillary Schubert polynomials in type \(C\) (Q2684901)

Schubert polynomials are certain polynomials indexed by permutations, which represent cohomology classes of Schubert varieties in the cohomology ring of the flag variety, forming an additive basis of said ring. Double Schubert polynomials are polynomials in two alphabets of variables which then represent the equivariant cohomology classes of Schubert varieties in the flag variety. Ikeda, Mihalcea, and Naruse introduced double Schubert polynomials of types \(B\), \(C\), and \(D\), which represent equivariant cohomology classes in the flag varieties of the corresponding types [\textit{T. Ikeda} et al., Adv. Math. 226, No. 1, 840--886 (2011; Zbl 1291.05222)]. Anderson and Fulton then defined the notion of a vexillary signed permutation in these types and showed that the corresponding double Schubert polynomials are given by certain Schur-Pfaffian formulas [\textit{D. Anderson} and \textit{W. Fulton}, ``Degeneracy loci, Pfaffians, and vexillary signed permutations in types B, C, and D'', Preprint, \url{arXiv:1210.2066}; Compos. Math. 154, No. 8, 1746--1774 (2018; Zbl 1412.14034)]. The goal of the paper at hand is to give a tableau formula for the type \(C\) double Schubert polynomials associated to vexillary signed permutations. The rough steps for doing this are as follows: introduce a certain type of tableaux corresponding to a certain type of partition; introduce a function associated to the partition defined in terms of tableaux; and finally show that these functions satisfy the Schur-Pfaffian formula of Anderson and Fulton. This then implies that the double Schubert polynomials of type \(C\) are equal to the functions associated to the partitions, and so satisfy a certain formula in terms of tableaux. Let us give more detail. The authors introduce marked shifted tableaux corresponding to flagged strict partitions. Here, a flagged strict partition is a sequence of strictly decreasing numbers (the strict partition) with an additional sequence of non-negative integers (the flagging). A marked shifted tableau is then the sort of tableau that corresponds to these; they have three different types of entries, which obey certain rules. Associated to a flagged strict partition we have a flagged factorial \(Q\)-function, which is a polynomial defined by summing over terms corresponding to marked shifted tableau for the flagged strict partition. When the flagging is zero, these correspond to the factorial \(Q\)-functions of \textit{V. N. Ivanov} [Zap. Nauchn. Semin. POMI 307, 99--119, 281--282 (2005; Zbl 1073.05069); translation in J. Math. Sci., New York 131, No. 2, 5495--5507 (2005)]. Their first main result is a Schur-Pfaffian formula for the flagged Schur \(Q\)-function associated to a given flagged strict partition (Theorem 22). The upshot is that this is the same formula as Anderson and Fulton give for the type \(C\) double Schubert polynomial associated to a vexillary signed permutation. Indeed, for every vexillary signed permutation there is an associated flagged strict partition, and the result shows that the double Schubert polynomial corresponding to the former equals the flagged Schur \(Q\)-function corresponding to the latter (Theorem 24). A final application of Theorem 22 is a new tableau formula for the factorial \(Q\)-functions of Ivanov.