Automorphism group computation and isomorphism testing in finite groups (Q1401150)

The authors generalize the automorphism group algorithm of \textit{E. A. O'Brien} [Computational algebra and number theory. Math. Appl., Dordr. 325, 83-90 (1995; Zbl 0836.20002)] and \textit{M. J. Smith} [Thesis, Australian National University (1994)] to the case of nonsolvable groups: Automorphisms of the radical factor are obtained via the normalizer in a suitable permutation representation. Then automorphisms are lifted along a characteristic series, similar to the approach of \textit{J. J. Cannon, B. C. Cox} and \textit{D. F. Holt} [J. Symb. Comput. 31, No. 1-2, 149-161 (2001; Zbl 0984.20002)] for subgroups. By choosing a particular characteristic series the authors are able to simplify the calculation of compatible pairs. The test for automorphisms that lift is performed by a backtrack search in a suitable permutation representation. The paper closes with an investigation of the algorithm's performance. The worst case is that of \(p\)-groups, for which an improved algorithm recently has been proposed by \textit{B. Eick, C. R. Leedham-Green} and \textit{E. A. O'Brien} [Commun. Algebra 30, No. 5, 2271-2295 (2002; Zbl 1006.20001)].