Theta lifting of unitary lowest weight modules and their associated cycles (Q1766399)

Consider reductive dual pairs of the form \((G,G')=(O(p,q),Sp(2n,\mathbb R)),(U(p,q),U(m,n))\) and \((Sp(p,q),O^*(2n))\) in the stable range, with \(G'\) the smaller member. In this paper, the authors study the theta lifts of unitary lowest weight modules of \(G'\), in particular the geometry of their associated varieties. Since by \textit{J.-S. Li} [Invent. Math. 97, No.2, 237-255 (1989; Zbl 0694.22011)], unitarity is preserved by the Howe correspondence in the stable range, these are (very singular) unitary representations of \(G\). Let \(\mathfrak g=\mathfrak k \oplus \mathfrak s\) and \(\mathfrak g'=\mathfrak k' \oplus \mathfrak s'\) be the Cartan decompositions of the complexified Lie algebras of \(G\) and \(G'\), respectively. For any nilpotent \(K'_{\mathbb C}\)-orbit \(\mathcal O'\) in \(\mathfrak s'\), \textit{K. Nishiyama, H. Ochiai} and \textit{C.-B. Zhu} [Theta lifting of nilpotent orbits for symmetric pairs, Trans. Am. Math. Soc., to appear] have defined in a natural way the theta lift \(\theta(\mathcal O')\), a nilpotent \(K_{\mathbb C}\)-orbit in \(\mathfrak s\). If \(\pi'\) is a unitary lowest weight module of \(G'\) then its associated variety \(\mathcal {AV}(\pi')\) consists of the closure of a single such orbit, and its associated cycle \(\mathcal {AC}(\pi')\) must be an integral multiple thereof. The main result is that the associated variety of the theta lift of \(\pi'\) is the theta lift of the associated variety of \(\pi'\), and the multiplicity in the associated cycles is preserved under the lifting. Moreover, a formula for this multiplicity is given. If (as is almost always the case) \(\pi'\) occurs in the correspondence for a dual pair \((G(k),G')\) with \(G(k)\) compact as the theta lift of a representation \(\sigma\) of \(G(k)\), then this is a simple formula in terms of the dimension of \(\sigma\). Generalizing the first part of the theorem, the authors show that if \(\rho'\) is any irreducible admissible (not necessarily unitary) representation of \(G'\) whose associated variety is irreducible (hence the closure of a single nilpotent orbit \(\mathcal O'\)), then the associated variety of the theta lift of \(\rho'\) is contained in the closure of \(\theta(\mathcal O')\). In addition, back in the setting of \(\pi'\) a unitary lowest weight module with theta lift \(\pi\), the authors obtain a formula for the \(K\)-structure of \(\pi\) in terms of the \(K'\)-structure of \(\pi'\).