On distinguished orbits of reductive representations (Q405906)

Let \(G\) be a real reductive Lie group and let \(\tau :G\to\text{GL}(V)\) be a real reductive representation of \(G\) with (restricted) moment map \(m_{\mathfrak{g}} :V\setminus\{0\}\to\mathfrak{g}\). The author introduces the notion of nice space of a real reductive representation to study the problem of how to determine if a \(G\)-orbit is distinguished (i.e. it contains a critical point of the norm squared of \(m_{\mathfrak{g}}\)). He gives an elementary proof of the well-known convexity theorem of Atiyah-Guillemin-Sternberg in the present context and uses it to give an easy-to-check sufficient condition for a \(G\)-orbit of an element in a nice space to be distinguished. In the case where \(G\) is algebraic and \(\tau\) is a rational representation, the above condition is also necessary (making heavy use of recent results of Michael Jablonski), obtaining a generalization of Nikolayevsky's nice basis criterion. The author also provides useful characterizations of nice spaces in terms of the weights of \(\tau\). Finally, some applications to ternary forms are presented.