Kazhdan-Lusztig coefficients for an affine Weyl group of type \(\widetilde B_2\). (Q555548)

In their famous paper [in Invent. Math. 53, 165-184 (1979; Zbl 0499.20035)], \textit{D. Kazhdan} and \textit{G. Lusztig} defined, for any two elements \(y,w\) of a Coxeter system \(W=(W,S)\), a polynomial \(P_{y,w}\) in one indeterminate \(q\) with integer coefficients. The inequality \(\deg P_{y,w}\leqslant\frac 12(\ell(w)-\ell(y)-1)\) holds in general when \(y<w\) under the Bruhat order. Denote by \(\mu(y,w)\) the coefficient of \(q^{\frac 12(\ell(w)-\ell(y)-1)}\) in \(P_{y,w}\) and call it the leading coefficient of \(P_{y,w}\). In his paper [Represent. Theory 1, 25-30 (1997; Zbl 0895.20030)], \textit{G. Lusztig} computed the leading coefficients for some Kazhdan-Lusztig polynomials of the affine Weyl group \(W(\widetilde B_2)\). In the paper under review, the author computes the coefficients \(\mu(u,w)\) for \(W(\widetilde B_2)\) in more cases. Let \(a\colon W\to\mathbb N\) be the function on \(W\) defined by \textit{G. Lusztig} [in Proc. Symp., Kyoto and Nagoya/Jap. 1983, Adv. Stud. Pure Math. 6, 255-287 (1985; Zbl 0569.20032)]. Let \(c_i=\{w\in W\mid a(w)=i\}\) for \(i\in\mathbb N\). Then \(W(\widetilde B_2)\) has four two-sided cells \(c_0\), \(c_1\), \(c_2\) and \(c_4\) [see loc. cit., Zbl 0569.20032]. Then the main results of the paper are as follows. (1) For any \(u\leqslant w\) in \(W(\widetilde B_2)\) with \((u,w)\notin(c_4\times c_4)\cup(c_4\times c_1)\cup(c_4\times c_2)\), the value \(\mu(u,w)\) is determined and displayed concretely. When \((u,w)\in c_4\times c_4\), the value \(\mu(u,w)\) can be computed by using a formula in the proof of \textit{L. Scott} and \textit{N.-H. Xi} [Sci. China, Math. 53, No. 8, 1919-1930 (2010; Zbl 1225.20003), Theorem 3.1]. (2) Using the values of \(\mu(u,w)\), the author verifies a conjecture of Lusztig on distinguished involutions in the case of \(W=W(\widetilde B_2)\) and proves that the \(W\)-graph of type \(\widetilde B_2\) is non-locally finite. (3) By computing some polynomials \(b_{\lambda,\lambda'}\) related to the \(P_{y,w}\)'s, the author modifies a conjectural formula of \textit{G. Lusztig} [in loc. cit., Zbl 0895.20030].