Parabolic double cosets in Coxeter groups (Q1700787)

Summary: Parabolic subgroups \(W_I\) of Coxeter systems \((W,S)\), as well as their ordinary and double quotients \(W / W_I\) and \(W_I \;W / W_J\), appear in many contexts in combinatorics and Lie theory, including the geometry and topology of generalized flag varieties and the symmetry groups of regular polytopes. The set of ordinary cosets \(w W_I\), for \(I \subseteq S\), forms the Coxeter complex of \(W\), and is well-studied. In this article we look at a less studied object: the set of all double cosets \(W_I w W_J\) for \(I, J \subseteq S\). Double cosets are not uniquely presented by triples \((I,w,J)\). We describe what we call the lex-minimal presentation, and prove that there exists a unique such object for each double coset. Lex-minimal presentations are then used to enumerate double cosets via a finite automaton depending on the Coxeter graph for \((W,S)\). As an example, we present a formula for the number of parabolic double cosets with a fixed minimal element when \(W\) is the symmetric group \(S_n\) (in this case, parabolic subgroups are also known as Young subgroups). Our formula is almost always linear time computable in \(n\), and we show how it can be generalized to any Coxeter group with little additional work. We spell out formulas for all finite and affine Weyl groups in the case that \(w\) is the identity element.