A limit theorem for shifted Schur measures (Q1885557)

The Schur \(Q\)-functions were introduced by Schur in his analysis of the projective representations of the symmetric group. These functions have a combinatorial theory analogous to the combinatorial theory of Schur functions; such a theory uses shifted Young tableaux and includes a shifted version of the Robinson-Schensted-Knuth algorithm; see \textit{B. E. Sagan} [J. Comb. Theory, Ser. A 45, 62--103 (1987; Zbl 0661.05010)]. Let \(\mathcal{D}\) be the set of all partitions into distinct parts. For \(\lambda\in\mathcal{D}\) denote by \(Q_{\lambda}(x)\) the corresponding \(Q\)-Schur function and set \(P_{\lambda}(x)=\frac{1}{2^{\ell(\lambda)}}Q_{\lambda}(x)\), where \(\ell(\lambda)\) is the length of \(\lambda\). The first result is an identity for the sum \( \sum_{\lambda_1\leq h,\,\lambda\in\mathcal{D}} Q_{\lambda}(x)P_{\lambda}(y)\) analogous to a result for Schur functions proved by \textit{I. M. Gessel} [ibid. 53, No. 2, 257--285 (1990; Zbl 0704.05001)]. It is obtained by using the representation of the \(Q\)-Schur functions as Pfaffians and a summation formula of \textit{M. Ishikawa} and \textit{M. Wakayama} [Linear Multilinear Algebra 39, No. 3, 285--305 (1995; Zbl 0830.15011)]. Then the authors introduce the probability measure that assigns to each \(\lambda\in\mathcal{D}\) the probability \( \mathbf{P}({\lambda})= \frac{1}{Z}Q_{\lambda}(x)P_{\lambda}(y)\) where \(Z\) is a normalization constant. It is called the shifted Schur measure and is analogous to the much-studied Schur measure. Then they specialize the measure by choosing the first \(m\) coordinates of \(x\) and the first \(n\) coordinates of \(y\) equal to \(\alpha\) (\(0<\alpha <1\)); this is called \(\alpha\)-specialization. The main result of the paper is the following limit law. Assume that \(\tau=m/n\) is constant and satisfies \(\alpha^2<\tau<\alpha^{-2}\). Let \(\mathbf{P}_\sigma\) denote the \(\alpha\)-specialized shifted Schur measure with \(\tau\) satisfing this condition. Then there exist constants \(c_1=c_1(\alpha,\tau)\) and \(c_2=c_2(\alpha,\tau)\) such that \[ \lim_{n\rightarrow\infty}\mathbf{P}_\sigma\left(\frac{\lambda_1-c_1n}{c_2n^{1/3}}<s\right)=F_2(s) \] where \(F_2\) is the limiting distribution of the largest eigenvalue (suitably centered and normalizd) in the Gaussian unitary ensamble [see \textit{C. A. Tracy} and \textit{H. Widom}, Commun. Math. Phys. 159, No. 1, 151--174 (1994; Zbl 0789.35152)]. This limit law coincides with that obtained by \textit{K. Johansson} for the \(\alpha\)-specialization of the Schur measure [ibid. 209, No. 2, 437--476 (2000; Zbl 0969.15008)].