Characters, dual pairs, and unitary representations (Q1261119)

The author considers preservation of unitarity under Howe's correspondence [c.f. \textit{R. Howe}, J. Am. Math. Soc. 2, 535-552 (1989; Zbl 0716.22006)], and the computation of the associated varieties and wave-front set of unitary representations. Let \(W\) be a finite-dimensional vector space over the reals, with a nondegenerate symplectic form \(\langle , \rangle\), and let \(G\), \(G'\) be an irreducible dual pair in the symplectic group \(S_ p= S_ p(W)\). Let \(\mathcal G\), \(\mathcal G'\) denote the Lie algebras of \(G\), \(G'\), respectively, and let \({\mathcal K}^*\) be the dual of \(\mathcal K\). Also let \(\widetilde G\) be the preimage of \(G\) in the metaplectic group \(\widetilde S_ p\) under the covering map \(\widetilde S_ p\to S_ p\). If \(\tau_{\mathcal G}: W\to {\mathcal G}^*\), \(\tau_{\mathcal G}': W\to{\mathcal G}'{}^*\) are the canonical moment maps defined by \(\tau_{\mathcal K}(x)={1\over 4}\langle x(w),w\rangle\) for \(w\in W\) and \(x\in{\mathcal K}\), these maps extend canonically to the complexifications \(\tau_{\mathcal G}: W_{\mathbb{C}}\to{\mathcal G}^*_{\mathbb{C}}\), \(\tau_{\mathcal G'}: W_{\mathbb{C}}\to {\mathcal G}_{\mathbb{C}}'{}^*\). The following theorem on the orbit structure of these maps is stated: If \(O\subset{\mathcal G}_{\mathbb{C}}^*\cap \tau_{\mathcal G}(W_{\mathbb{C}})\) is a nilpotent coadjoint orbit, then there is a unique coadjoint orbit \(O'\subset{\mathcal G}_{\mathbb{C}}'{}^*\) such that \(\tau_{\mathcal G'}(\tau_{\mathcal G}^{-1}(\overline O))=\overline{O'}\), where \(\overline A\) denotes the closure of \(A\). The above theorem is a result of joint work by the author with A. Daszkiewicz and W. Kraśkiewicz. A strong criterion for preservation of unitarity under Howe's correspondence is established, Theorem 3.1. Also, the following theorem which is the intersection of the main results of the paper is proved: Let \(\Pi\) be an irreducible unitary representation of \(\widetilde G\). Suppose that the character \(\Theta_ \Pi\) has the rate of growth \(\gamma\geq 0\) such that \(d'> \gamma(r-1)+ r-1\), and that there is a vector \(u\) in the Harish-Chandra module of \(\Pi\) such that \[ \int_{\widetilde G} (u,\Pi(g)u)(w(g)v,v)dg\geq 0,\quad v\in H^ \infty_ \omega \] and this integral is nonzero for some \(v\). Then \(\Pi\in{\mathfrak R}(G,\omega)\) and the corresponding representation \(\Pi'\in {\mathfrak R}(G',\omega)\) through Howe's correspondence is unitary. The readers are referred to the paper for notions and symbols not defined here.