Cantor dynamics of renormalizable groups (Q2076080)

A \emph{renormalizable} group is a group \(\Gamma\) admitting a proper self-embedding \(\phi\colon\Gamma\hookrightarrow\Gamma\) whose image has finite index. If \(K\coloneqq\bigcap_{n\ge0}\phi^n(\Gamma)\) is a finite subgroup, what can one deduce about \(\Gamma\)? Among finitely generated groups, the main known examples are nilpotent groups with dilatations, so it might be believed that they are the only ones; this appears as a question in [\textit{V. Nekrashevych} and \textit{G. Pete}, Groups Geom. Dyn. 5, No. 1, 139--167 (2011; Zbl 1236.20031)]. The present article answers partially this question, as follows. From the renormalization \(\phi\), there is a natural action of \(\Gamma\) on a locally finite rooted tree with vertex set the cosets \(\bigsqcup_{n\ge0}\Gamma/\phi^n(\Gamma)\) and edges given by inclusion. Let \(\widehat\Gamma\) denote the closure of \(\Gamma\) in the automorphism group of this tree (with the usual topology on functions). There is naturally an induced renormalization \(\widehat\phi\colon\widehat\Gamma\to\widehat\Gamma\), and (Theorem 1.3) if \(\Gamma\) is finitely generated then the image of \(\widehat\phi\) is open. In analogy with \(K\), introduce the limit \(\mathscr D=\bigcap_{n\ge0}\widehat\phi^n(\widehat\Gamma)\). The partial answer may then be stated (Theorems 1.1 and 1.5): assume \(\Gamma\) is finitely generated. If \(\mathscr D\) is finite, then \(\Gamma/\text{Core}(K)\) is virtually nilpotent; and is actually nilpotent if \(\mathscr D\) is trivial. In particular, if \(K\) and \(\mathscr D\) are both finite then \(\Gamma\) is virtually nilpotent. The paper contains, in its last section, an extensive list of open problems on renormalizations.