Compact actions, retract theory and prime ideals (Q351843)

Let \(G\) be a Lie group and \((L^1(G),\ast)\) the algebra of integrable functions on \(G\), with respect to a given Haar measure. Here, \(\ast\) denotes the convolution. An ideal \(I\subset L^1(G)\) is said to be prime if, for any two-sided ideals \(I_1\) and \(I_2\) of \(L^1(G)\) such that \(I_1\ast I_2\subset I\), \(I_1\subset I\) or \(I_2\subset I_2\) holds. For nilpotent connected and simply connected Lie groups, \textit{J. Ludwig} [Math. Ann. 262, 287--304 (1983; Zbl 0489.22009)] showed that the closed prime ideals of \(L^1(G)\) coincide with the kernels of the irreducible unitary representations of \(G\). Later, \textit{D. Poguntke} [Math. Ann. 269, 431--467 (1984; Zbl 0536.43012); Proc. Cent. Math. Anal. Aust. Natl. Univ. 16, 247--261 (1988; Zbl 0675.22004)] considered the action of an abelian compact group \(K\) on a nilpotent Lie group and characterized the \(K\)-prime ideals. In this paper, the authors consider the action of a non-abelian compact Lie group of the automorphism groups of a nilpotent Lie group \(G\), assumed to be connected and simply connected. They determine the \(K\)-prime ideals of the Schwartz space \(S(G)\) and of \(L^1(G)\), with some restriction in the case of \(L^1(G)\). Namely, they show the density of \(S(G)\) in the kernels of \(K\)-orbits and they characterize the corresponding \(K\)-prime ideals. To do so, they prove a global retract theorem for the layer of generic elements in the sense of \textit{J. Ludwig} and \textit{H. Zahir} [Lett. Math. Phys. 30, No. 1, 23--34 (1994; Zbl 0798.22004)], which is at the heart of the paper. For arbitrary intermediate layers, the problem remains open.