Affine Brauer category and parabolic category \(\mathcal{O}\) in types \(B, C, D\) (Q2272960)

The authors introduce an \textit{affine Brauer category} \(\mathcal{AB}\) of diagrams (working over a commutative ring \(\kappa\) with 1 and having invertible element 2). Compared to the ordinary Brauer category \({\mathcal B}\), the objects are again represented by natural numbers, but, in addition to the generating morphisms and relations for \({\mathcal B}\), one additional morphism and two additional relations are added to define \(\mathcal{AB}\). Also considered are certain specializations of these categories: \({\mathcal B}(\omega_0)\), \(\mathcal{AB}(\omega_0)\), and \(\mathcal{AB}(\omega)\), where \(\omega_0 \in \kappa\) and \(\omega\) is an ``admissible'' sequence of elements in \(\kappa\). A key goal of the paper is to determine \(\kappa\)-bases for spaces of homomorphisms \(\operatorname{Hom}_{{\mathcal C}}(\mathsf{m},\mathsf{s})\), where \({\mathcal C}\) is one of these categories and \(\mathsf{m}\) and \(\mathsf{s}\) are objects in the category associated to the integers \(m\) and \(s\). Working over the complex numbers and using a monoidal functor from \({\mathcal B}\) to the category of finite-dimensional modules for an orthogonal (\({\mathfrak so}_N\)) or symplectic (\({\mathfrak sp}_N\)) Lie algebra, the authors identify a basis for any \({\mathcal B}\)-homomorphism space in terms of equivalence classes of certain Brauer diagrams. For an element \(w_0 \in \kappa\), this gives a basis for Hom spaces over \({\mathcal B}(\omega_0)\), recovering the results of \textit{G. Lehrer} and \textit{R. B. Zhang} [J. Eur. Math. Soc. 17, No. 9, 2311--2351 (2015; Zbl 1328.14079)]. For the category \(\mathcal{AB}\), the authors introduce \textit{dotted Brauer diagrams} and identify Hom spaces over \(\mathcal{AB}\) (or its specializations) in terms of equivalence classes of certain ``normally ordered'' dotted Brauer diagrams. In this case, key to the identification is the construction of a monoidal functor from the category \(\mathcal{AB}\) to the category of endofunctors of the category of modules over \(\mathfrak{so}_N\) or \(\mathfrak{sp}_N\). The authors use these ideas to identify an endomorphism algebra over \(\mathcal{AB}(\omega)\) with an affine Nazarov-Wenzl algebra. The authors further introduce a \textit{cyclotomic Brauer category} \(\mathcal{CB}^f\) (and specialization \(\mathcal {CB}^f(\omega)\) for certain \(\omega\)) as the quotient of \(\mathcal {AB}\) by the (right tensor) ideal generated by a monic polynomial \(f\) over \(\kappa\). From the aforementioned functor on \(\mathcal{AB}\), the authors construct a functor from \(\mathcal{AB}\) to a module category \({\mathcal O}^{\mathfrak p} \subset {\mathcal O}\) for certain parabolic subalgebras \({\mathfrak p}\) of \(\mathfrak{so}_N\) or \(\mathfrak{sp}_N\) with BGG category \({\mathcal O}\). For appropriate choice of polynomial \(f\), this functor is shown to factor through \(\mathcal{CB}^f\), allowing the authors to identify Hom spaces over \(\mathcal{CB}^f(\omega)\) for certain ``\({\mathbf u}\)-admisible'' \(\omega\). As part of that process, for these \(\omega\), endomorphism algebras over \(\mathcal{CB}^f(\omega)\) are identified with cyclotomic Nazarov-Wenzl algebras. This then leads to an identification of cyclotomic Nazarov-Wenzl algebras with a category \({\mathcal O}\) endomorphism algebra, extending work of \textit{M. Ehrig} and \textit{C. Stroppel} [Adv. Math. 331, 58--142 (2018; Zbl 1432.16022)]. With that identification, one could compute decomposition numbers for cyclotomic Nazarov-Wenzl algebras.