Localization operators for generalized Weyl-Heisenberg group (Q2210467)

\textit{P. Boggiatto} and \textit{M. W. Wong} [Integral Equations Oper. Theory 49, No. 1, 1--10 (2004; Zbl 1072.47046)] investigate the time-frequency localization operators \(L_{F,\phi,\psi}\) associated with two windowed functions \(\phi, \psi\) on the Weyl-Heisenberg group and obtain the boundedness and compactness of \(L_{F,\phi,\psi}\). Motivated by their works the authors investigate \(L_{F,\phi,\psi}\) corresponding to a square-integrable representation on the generalized Weyl-Heisenberg group. Let \(H\) be a locally compact group and \(K\) be a locally compact abelian group. Let \(\tau : H\to \mathrm{Aut}(K)\), \(h\mapsto \tau_h\), be a continuous homomorphism and \(\hat{\tau} : H\to \mathrm{Aut}(\hat{K})\) be given by \(\hat{\tau}_h(\omega)=\omega\circ\tau_{h^{-1}}\). Let \(G_\tau=H\times_\tau K\). We define \(\theta : G_\tau\to \mathrm{Aut}(\hat{K}\times{\mathbb T})\), \((h,k)\mapsto \theta_{(h,k)}\) by \(\theta_{(h,k)}(\omega, z)=(\hat{\tau}_h(\omega), \hat{\tau}_h(\omega)(k)z)\). Then the generalized Weyl-Heisenberg group \({\mathbb H}(G_\tau)\) associated with \(G_\tau\) is defined by \(G_\tau\times_{\theta}(\hat{K}\times{\mathbb T})\). The classical Heisenberg group and the Weyl-Heisenberg group are given as \({\mathbb R}^n\times_\theta({\mathbb R}^n\times{\mathbb T})\) and \(K\times_\theta(\hat{K}\times{\mathbb T})\) respectively. The representation \(\pi : {\mathbb H}(G_\tau)\to U(L^2(\hat{K}))\) given by \(\pi(h,k,\omega,z)f(\xi)\) \(=\) \(\delta_H^{-1/2}(h)z\xi(k)\overline{\omega(k)}f((\xi\overline{\omega})_{h^{-1}})\) is square-integrable if and only if \(H\) is compact and is irreducible if \(H\) is compact. The authors obtain the boundedness and compactness of the localization operator \(L_{F,\phi,\psi}\) for some \(\phi\in L^p(\hat{K})\), \(\psi\in L^q(\hat{K})\) and \(F\in L^r( {\mathbb H}(G_\tau))\). They also treat the representation \(\tilde{\pi} : {\mathbb H}(G_\tau)\to U(L^2(K))\) given by \(\tilde{\phi}(h,k,\omega,z)f(k')\) \(=\) \(\delta_H^{1/2}(h)z\omega(k')f((\tau_{h^{-1}}(k'k))\), which is unitarily equivalent to \(\pi\), and give some examples of \(L_{F,\phi,\psi}\).