Fixed point polynomials of permutation groups. (Q1953509)

Let \(G\) be a group acting faithfully on a set \(\Omega\) of cardinality \(n<\infty\). In this paper the authors define the `fixed-point polynomial', \(P_{G,\Omega}(x):=\sum_{i=1}^n f_ix^i\), where \(f_i\) is the number of elements of \(G\) that fix exactly \(i\) points of \(\Omega\). The authors prove a number of results connecting the behaviour of the roots of \(P_{G,\Omega}\) to properties of the action of \(G\) on \(\Omega\). A typical example of such a result is the following: Let \((G_n)_{n\in\mathbb N}\) be a family of groups such that, for each \(n\), \(G_n\) acts faithfully and primitively on \(\Omega_n\), a set of cardinality \(n\). Suppose moreover that \(\mathrm{Alt}(n)\not\leq G_n\) for all \(n\). Then, for any \(\rho\in(0,1)\), \(P_{G_n,\Omega_n}\) has a proportion \(1-o(1)\) of its roots in the annulus \[ \bigl\{\alpha\in\mathbb C\mid 1-\rho\leq |\alpha|\leq\tfrac{1}{1-\rho}\bigr\} \] as \(n\to \infty\). This result is proved using theorems from the study of analytic functions and from permutation group theory. The authors also explicitly calculate \(P_{G,\Omega}(x)\) for various families of permutation groups. Thus, for example, \[ \begin{aligned} P_{\mathrm{Sym}(n),[n]}(x)&=n!\sum_{i=0}^n\frac{(x-1)^i}{i!};\\ P_{\mathrm{Alt}(n),[n]}(x)&=\tfrac12\left(P_{\mathrm{Sym}(n),[n]}(x)+(x-1)^n+n(x-1)^{n-1}\right);\\ P_{G,\Omega}(x)&=x^{|\Omega|}+(|G|-|\Omega|)x+|\Omega|-1\end{aligned} \] where, in the last example, the action of \(G\) on \(\Omega\) is Frobenius. The authors remark that, for the first two of these examples, the fixed-point polynomial is irreducible over \(\mathbb Q\) while in the last example \(P_{G,\Omega}\) is irreducible over \(\mathbb Q\) whenever \(|\Omega|\) is prime. More results of a similar nature can be found in the PhD thesis of the first author.