The generalized Borwein conjecture. II: Refined \(q\)-trinomial coefficients (Q1412668)

Peter Borwein conjectured that the polynomials \(A_n(q)\), \(B_n(q)\), and \(C_n(q)\) defined by \[ \prod_{k=1}^n (1-q^{3k-2}) (1-q^{3k-1}) =A_n(q^3)-q B_n(q^3)-q^2C_n(q^3) \] have nonnegative coefficients. \textit{D. M. Bressoud} [Electron. J. Comb. 3, Research paper R4 (1996; Zbl 0856.05007); printed version J. Comb. 3, 133-146 (1996)] observed that for \[ G(N,M;\alpha,\beta,K)=\sum_{j=-\infty}^{j=\infty} (-1)^jq^{{1\over 2}Kj(( \alpha+\beta)j+\alpha-\beta)}{M+N \brack N-Kj}, \] \(A_n(q)= G(n,n;4/3,5/3,3)\), \(B_n(q)=G(n+1,n-1;2/3,7/3,3)\), and \(C_n(q)=G(n+1,n-1;1/3,8/3,3)\). Bressoud generalized Borwein's conjecture as follows: Let \(K\) be a positive integer and \(N\), \(M\), \(\alpha K\) and \(\beta K\) be nonnegative integers such that \(1\leq \alpha+\beta \leq 2K-1\) (strict inequalities when \(K=2\)) and \(\beta-K \leq N-M\leq K-\alpha\). Then \(G(N,M;\alpha,\beta,K)\) is a polynomial of \(q\) with nonnegative coefficients. This paper is a continuation of \textit{S. O. Warnaar} [Contemp. Math. 291, 243-267 (2001; Zbl 0994.05009)] that used the Burge transform to settle Bressoud's conjecture for two classes of problems. For a sequence of \(n+1\) positive integers \(a_0,a_1,\dots,a_n\), set \(d = a_0+a_1+\cdots+a_n\), and \(\mu = {2\over 3}(4^{d-1}-1)-\sum_{i=1}^n 4^{a_i+\cdots+a_n-2}.\) The paper proves the following results on Bressoud's conjecture: For \(n\geq 0\), \(a_0\geq 0\), \(a_1,\dots,a_n\geq 1\), \(a_n\geq 2-\delta_{n,0}\) integers, \(M\) nonnegative integer, and \(\alpha, \beta, K\) integers such that \(0\leq \alpha, \beta\leq K\), \[ G(M,M;(\alpha+\mu K)/2^{d-1}, (\beta+\mu K)/2^{d-1}, 2^{d-1}K)\geq 0; \] for a positive integer \(K\) \[ G(M,M;((2\mu +1) K-1)/2^{d}, ((2\mu +1) K+1)/2^{d}, 2^{d-1}K)\geq 0; \] and for \(a,b\) coprime integers such that \(\mu a < b < (\mu+1) a\) \[ G(M,M;b/2^{d-1}, (b+1/a)/2^{d-1}, 2^{d-1}a)\geq 0. \]