The intersection of the subgroups of finite index in Baumslag-Solitar groups. (Q1957017)

For a group \(G\) let \(\sigma(G)\) denote the intersection of all finite-index normal subgroups of \(G\). Then \(\sigma(G)=1\) if and only if \(G\) is residually finite. The article first states results concerning \(\sigma(G)\) for certain classes of groups that follow from earlier papers. A result for \(\sigma(G)\) for HNN-extensions follows from: [\textit{D. I. Moldavanskii}, Ukr. Math. J. 44, No. 6, 758-760 (1992); translation from Ukr. Mat. Zh. 44, No. 6, 842-845 (1992; Zbl 0788.20014)]. Theorem. Let \(G\) be a group, \(\varphi\) an injective endomorphism of \(G\), and let \(G(\varphi)=\langle G,t\mid t^{-1}gt=g\varphi,\;(g\in G)\rangle\) be the HNN-extension of \(G\) relative to \(\varphi\). Then \(\sigma(G(\varphi))\) coincides with the normal closure in \(G(\varphi)\) of the intersection of all \(\varphi\)-compatible finite-index subgroups of \(G\). (A subgroup \(H\) of \(G\) is called \(\varphi\)-compatible if \(H\varphi=G\varphi\cap H\).) A result for \(\sigma(G)\) for generalised (amalgamated) free products follows from [\textit{D. I. Moldavanskii}, ``The intersection of the subgroups of finite-index in some generalized free products of groups.'' Vestn. Ivanovsk. Gos. Un-ta. Ser. Estestv., Obshchestv. Nauki, No. 2, 114-122 (2008)]: Theorem. Let \(A,B\) be residually finite groups and let \(H\) be a proper subgroup of \(A\), \(K\) a proper central subgroup of \(B\), \(\varphi\colon H\to K\) an isomorphism, and let \(G\) be the generalised (amalgamated) free product \(\langle A*B\mid H=K,\;\varphi\rangle\). Then \(\sigma(G)\) coincides with the normal closure in \(G\) of the mutual commutator set \([H_1,B]\), where \(H_1\) is the intersection of all finite-index subgroups of \(A\) containing the subgroup \(H\). The present paper provides a similar result for \(\sigma(G)\) for the Baumslag-Solitar groups: Theorem 1. Let \(G(m,n)\) be the Baumslag-Solitar group \(\langle a,b\mid a^{-1}b^ma=b^n\rangle\) where \(m,n\) are nonzero integers. Then \(\sigma(G(m,n))\) coincides with the normal closure in \(G(m,n)\) of the set of commutators \([a^kb^da^{-k},b]\) where \(d=\gcd(m,n)\) and \(k\) takes all integer values. The known result that \(G(m,n)\) is residually finite if and only if \(|m|=1\) or \(|n|=1\) or \(|m|=|n|\) follows as a corollary. Thus \(\sigma(G(m,n))\) is trivial in these cases. Theorem 2 of the paper states that, in fact, this is the only case when \(\sigma(G(m,n))\) is finitely generated.