Nilpotent Lie groups: Fourier inversion and prime ideals (Q666643)

Let $G=\exp(\mathfrak{g})$ be a connected, simply connected, nilpotent Lie group of dimension $n$, all the irreducible unitary representations of $G$ are obtained (up to equivalence) by induction $\pi_l= \operatorname{Ind}_{P(l)}^G \chi_l$, for all $l \in \mathfrak{g}^*$, on the Hilbert space \[ \mathcal{H}_l= L^2(G/ P(l), \chi_l), \] where $P(l) = \exp(\mathfrak{p}(l))$ and $\mathfrak{p}(l)$ is an arbitrary polarisation. The corresponding representation $\pi_l$ on $L^1(G)$ is the kernel operator \[ (\pi_l(f)\xi)(g) = \int_{G/P(l)}F(l, g, u) \xi(u) du, \quad \text{with}\quad F(l, g, u)= \int_{P(l)} f(ghu^{-1})\chi_l(h) dh. \] R. Howe proved that for every $(\pi, \mathcal{H}_\pi)\in \widehat{G}$ and every smooth linear operator $a$ on $ \mathcal{H}_\pi$ there exists a Schwartz function $f_a \in \mathcal{S}(G)$ such that $\pi(f_a)= a$. He also showed that the mapping $a \mapsto f_a$ is linear and continuous with respect to the Fréchet topology of the space $\mathcal{B}^\infty(\mathcal{H}_\pi)$ of smooth linear operators on $\mathcal{H}_\pi$ and the Fréchet topology on $\mathcal{S}(G)$. In this paper, the authors give a generalisation of this result. In fact, they introduce a smooth variable (nilpotent) Lie algebra as $(\mathcal{B}, \mathfrak{g})$ such that for any $\beta \in \mathcal{B}$, $\mathfrak{g}_\beta= (\mathfrak{g} [., .]_\beta)$ there is a nilpotent Lie algebra and there exists a fixed Jordan-Hölder basis $\mathcal{Z}=\{Z_1, \cdots, Z_n\}$ for $\mathfrak{g} _\beta$ with $\mathcal{B}$ that is a smooth manifold. They associate a variable Lie group $\mathbb{G} = (G_\beta)_{\beta \in \mathcal{B}}$, which may be identified with the collection of Lie algebras $(\mathfrak{g}, [., .]_\beta)_{\beta \in \mathcal{B}}$, equipped with the corresponding Campbell-Baker-Hausdorff multiplications. They define for $r\in \mathbb{N}$, the space of (generalised) Schwartz functions $\mathcal{S}(\mathbb{R}^r, \mathcal{B}, G)\equiv \mathcal{S}(\mathbb{R}^r, \mathcal{B}, \mathfrak{g}) \equiv \mathcal{S}(\mathbb{R}^r, \mathcal{B}, \mathbb{R}^n)$, defined by a collection of seminorms. Given $M$, a sub-manifold of $\mathcal{B}\times \mathfrak{g}^*$, they introduce the adapted field of kernel functions $F= (F(\beta, l))_{(\beta, l)\in M}$ and the kernel operators on $L^2(G/P(\beta, l), \chi(\beta, l))$ by \[ \operatorname{op}_{F(\beta, l)}\xi(g)= \int_{G/P(\beta, l)}F(\beta, l)(g, x) \xi(x)d{\dot{ x}}, \] with $P(\beta, l)= \exp(\mathfrak{p}(\beta, l))$ the corresponding family of Vergne polarisation for the induced unitary representations $\pi(\beta, l)$. Let $I= \{ (j_1, k_1) < \cdots <(j_r, k_r)\}$ be an index set and $M$ a $G$-invariant sub-manifold of $\mathcal{B}\times \mathfrak{g}^*$ such that $ M_I:= M\cap(\mathcal{B}\times \mathfrak{g}^*)_I \not= \varnothing$. The authors prove the existence of an open nonempty relatively compact subset $\mathcal{M} \subset M_I$ with closure $\overline{\mathcal{M}} \subset M_I$ such that: For any adapted kernel function $F $, there is a function $f \in \mathcal{S}(\mathbb{R}^n, \mathcal{B}, G)$ such that $\pi(\beta, l)f(\alpha, (\beta, .)$ has $F(\alpha, (\beta, l), .,.)$ as an operator kernel for all $(\alpha, (\beta, l))\in \mathbb{R}^r\times M$. The map of the Fourier inversion theorem is continuous and given by \[ f(\beta, g) = \int_{\Sigma_\beta,I_{\max}}\operatorname{tr}(\pi(\beta, l)(g)\circ \operatorname{op}_{F(\beta, l)})|P_a(\beta, l)|dl, \ \forall g \in G, \] where $I_{\max}$ is a maximal index, $P_a(\beta, l)$ is the Pfaffian of the polynomial $Q(l)=\det (\langle l[Z_i, Z_j]_\beta\rangle_{i, j \in I_{\max}})$ and \[ \Sigma_{\beta,I_{\max}}= \{ (\beta, l)\quad l~ \in \operatorname{span}(Z_j, j \in I_{\max} )\text{ and } I_{\max}(\beta, l)= I_{\max}\}. \] As an application of this result, they provide a complete characterisation of a large class of invariant prime closed two-sided ideals of $L^1(G)$. Precisely, they show that every $\mathbb{G}$-prime ideal in $L^1(G)$ is the kernel of a $\mathbb{G}$-orbit.