Basic calculus of signed permutations. I: Length and number of inversions (Q1356118)

Several investigators (e.g., F. Brenti, J. Stembridge, V. Reiner) have calculated the exponential generating function for the sequence of polynomials \(\{W_n(X,Y,t)\}\) where each \(W_n(X,Y,t)\) gives the trivariate distribution on signed permutations of descent number with the number of letters of each sign: \[ \frac{(1-t)\exp ((t-1)X)}{-t+\exp ((t-1)(X+Y))}=\sum\limits_{n\geq 0}\frac 1{n!}W_n(X,Y,t). \] \textit{V. Reiner} [Discrete Math. 140, No. 1-3, 129-140 (1995; Zbl 0830.20025)] has calculated a \(q\)-analogue whose coefficients are combinatorially interpreted as the polynomial generating functions for descent number and Coxeter length. In this article, the authors offer different \(q\)-analogues of the formula above by replacing the ordinary exponential function first by the \(q\)-exponential \(e_q(u)=\frac 1{(u;q)_\infty }\), then by the \(Q\)-exponential \( E_Q(u)=(-u;Q)_\infty .\) The new coefficients are again generating polynomials for certain pairs of statistics on signed permutations, namely (des, coinv) and (des, inv). (Note: the ``inv'' of this paper is not the same as Coxeter length.) More generally, replacing the ordinary exponential function by a multibasic Bessel function leads to polynomial generating functions on ``signed multipermutations of order \(n\)''. The statistics of interest are natural generalizations of ``des'', ``inv'' and ``coinv''. The \( q\)- and \(Q\)-analogues described earlier are obtained as specializations of this result, as are several other well-known formulas. The proof of the main theorem is accomplished using an iterative procedure to show that the desired polynomial generating functions satisfy the correct recurrence. The same procedure is also used to give a different proof of Reiner's result. In a subsequent article [Electron. J. Comb. 4, No. 2, R5 (1997)], the authors extend their main result in a different way, using finite analogues of Bessel functions.