Nilpotent subalgebras of semisimple Lie algebras (Q1018111)

Let \(k\) be an algebraically closed field of characteristic \(p>0.\) Let \(G\) be a semisimple linear algebraic group over \(k\) with fixed maximal torus \(T.\) Let \(\Phi=\Phi\left( G,T\right) \) be its root system, \(\Phi^{+}\) a positive system in \(\Phi,\) and \(\left\{ \alpha_{1},\dots,\alpha_{\ell}\right\} \) the corresponding basis. Suppose that is irreducible. Then there is a unique root \(\beta\) of maximal height with respect to the chosen basis. Write \(\alpha _{1}^{\vee},\dots,\alpha_{\ell}^{\vee},\beta^{\vee}\) for the corresponding coroots to \(\alpha_{1},\dots,\alpha_{\ell},\beta\) respectively, and define integers \(m_{i}^{\prime},\,1\leq i\leq\ell\) by \(\beta^{\vee}=\sum m_{i}^{\prime}\alpha_{i}^{\vee}\) We say \(p\) is \textit{torsion} for \(\Phi\) if \(m_{i}^{\prime}=p\) for some \(i\). Additionally, we say \(p\) is torsion for \(G\) if it is torsion for some irreducible component of some root system or if \(p\) divides the order of the fundamental group of \(G\). If \(p\) is a non-torsion prime for \(G\) and \(U\) is a unipotent subgroup of \(G\), then \(G\) is contained in a Borel subgroup of \(G\). For \(\mathfrak{g}\) the Lie algebra for \(G\) and \(\mathfrak{n}\) a nil subalgebra of \(\mathfrak{g},\) it follows from this result that if \(p\) it non-torsion for \(G\) then \(\mathfrak{n}\) lies in a Borel subalgebra of \(G\). Conversely, J. P. Serre asked it there exists a nil subalgebra which is not contained in a Borel subalgebra in the case \(p\) is torsion for \(G\). This note provides examples of such subalgebras, examples which are presented in two cases. As an explicit class, consider \(G=\)SL\(_{mp}\) for some positive integer \(m\). Let \(E_{ij}\) be the elementary \(mp\times mp\) matrix whose \(\left( r,s\right) \) entry is \(\delta_{ir}\delta_{js},\) and let \(X=\sum_{j=0}^{m-1}\sum _{i=1}^{p-1}E_{jp+i,jp+i+1}\) and \(Y=\sum_{j=0}^{m-1}\sum_{i=1}^{p-1} E_{jp+i+1,jp+i}.\) Then \(X^{p}=Y^{p}=0\) and \(\left[ X,Y\right] =I,\) thus the Lie algebra generated by \(X\) and \(Y\) is nilpotent and there is not a Borel subalgebra containing \(X\) and \(Y\). For other simply connected \(G\) explicit examples such as this are given -- these ``Heisenberg-type subalgebras'' all have an \(X\) and a \(Y\) with \(X^{p}=Y^{p}=0\) and \(\left[ X,Y\right] \) a nontrivial element in the center of \(\mathfrak{g}.\) These results are generalized to groups isogenous to \(G.\) The other case deals with certain commutative subalgebras. These are explicitly constructed when \(p\leq5.\) In each the authors construct a 3-dimensional commutative restricted subalgebra of \(\mathfrak{g}\) spanned by three elements \(e,X,Y\) with \(e^{p}=X^{p}=Y^{p}=0\) which does not lie in a Borel subalgebra.