Geometry of phylogenetic group-based models (Q663592)

The main theorem of the paper shows that some algebraic varieties, called phylogenetic group-based models, arising from problems considered in algebraic statistics, are toric non-normal varieties. More precisely, let \(S\) be a finite set of states and let \(W\) be the complex vector space freely generated by the elements of \(S.\) Let \(G\) be a finite subgroup of permutations of the set \(S\) and let \(\widehat W\) be the maximal set of endomorphisms of \(W,\) invariant under the action of \(G.\) Let \(T\) be a tree with vertices \(V\) and edges \(E\). There is a copy of \(W\) attached to every vertex \(v\in V\) and a copy of \(\widehat W\) attached to every egdge \(e\in E.\) Such data describe a model of evolution in computational biology. Let \(L\) denote the set of leaves of the tree \(T\). There exists a rational map \[ \prod_{e\in E} \mathbb{P}(\widehat W_e) \dashrightarrow \mathbb{P}(\bigotimes_{l\in L} W_l), \] which image closure is called the \(G\)-model. It is known that for an abelian group \(G\) the \(G\)-model is a toric variety, possibly non-normal (cf. [\textit{S. N. Evans} and \textit{T. P. Speed}, Ann. Stat. 21, No. 1, 355--377 (1993; Zbl 0772.92012)] and [\textit{L. A. Székely, M. A. Steel} and \textit{P. L. Erdös}, Adv. Appl. Math. 14, No. 2, 200--216 (1993; Zbl 0794.05014)]). The main theorem of the paper under review shows that for any group \(G\) containing a normal abelian subgroup acting freely and transitively on the set \(S\) the \(G\)-model is non-normal toric variety. The proof uses the discrete Fourier transform and generalization of notions developed by \textit{W. Buczyńska} and \textit{J. A. Wiśniewski} [J. Eur. Math. Soc. (JEMS) 9, No. 3, 609--635 (2007; Zbl 1147.14027)]. The paper contains examples of groups \(G\) with normal \(G\)-models and non-normal ones. Moreover, based on work \textit{B. Sturmfels} and \textit{S. Sullivan} [Mich. Math. J. 57, 689--709 (2008; Zbl 1180.13040)], it is explained how to compute a polytope related to \(G\)-model for group \(G\) as in the main theorem.