In praise of tedious permutation (Q4425375)

Tests of the null hypothesis by repeated permutation of the observed data are brutally simple, valid on the basis of only a few assumptions, and adaptable to any test statistic of interest. The history of permutation tests is briefly reviewed, from the pioneering work of R. A. Fisher in the 1930s, to contemporary applications to human brain mapping by several independent groups. Permutation tests for a difference between two sets of brain images at voxel and cluster levels are described, and compared wherever possible to the corresponding theoretical tests. It is shown that nominal Type I error control by permutation is generally excellent; and that some interesting statistics, such as the ``mass'' of a suprathreshold voxel cluster, are more accessible to testing by permutation than theory. Some limitations of permutation tests are discussed. On balance, the case for permutation tests of the null hypothesis in brain mapping is strong. For the entire collection see [Zbl 0960.00021].