Invalid permutation tests

DOI10.1155/2010/769780MaRDI QIDQ978997zbMATH OpenOpenAlexWikidataFDO

Authors Aickin, Mikel

Publication date 25 June 2010

Published in International Journal of Mathematics and Mathematical Sciences (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://eudml.org/doc/232818

Mathematics Subject Classification ID

Foundations and philosophical topics in statistics (62A01) Nonparametric hypothesis testing (62G10)

Review text

Summary: Permutation tests are often presented in a rather casual manner, in both introductory and advanced statistics textbooks. The appeal of the cleverness of the procedure seems to replace the need for a rigorous argument that it produces valid hypothesis tests. The consequence of this educational failing has been a widespread belief in a ``permutation principle, which is supposed invariably to give tests that are valid by construction, under an absolute minimum of statistical assumptions. Several lines of arguments are presented here to show that the permutation principle itself can be invalid, concentrating on the Fisher-Pitman permutation test for two means. A simple counterfactual example illustrates the general problem, and a slightly more elaborate counterfactual argument is used to explain why the main mathematical proof of the validity of permutation tests is mistaken. Two modifications of the permutation test are suggested to be valid in a very modest simulation. In instances where simulation software is readily available, investigating the validity of a specific permutation test can be done easily, requiring only a minimum understanding of statistical technicalities.

Recommendations

• Exact and asymptotically robust permutation tests
• scientific article; zbMATH DE number 1978940
• Publication:4860267
• Permutation Tests for Linear Models
• Multivariate and multiple permutation tests

Cites work

• Concepts of nonparametric theory
• scientific article; zbMATH DE number 3145638 (Why is no real title available?)
• On the Behavior of Randomization Tests Without a Group Invariance Assumption
• PERMUTATION TESTS AND OTHER TEST STATISTICS FOR ILL-BEHAVED DATA: EXPERIENCE OF THE NINDS t-PA STROKE TRIAL

Cited in

(13)

• Interaction testing: residuals-based permutations and parametric bootstrap in continuous, count, and binary data
• The split sample permutation \(t\)-tests
• Permutation test for validation of binary regression models
• Some remarks about the number of permutations one should consider to perform a permutation test
• Permutation Tests Using Estimated Distribution Functions
• Permutation Tests for Correlation in Regression Errors
• Identifying settings when permutation tests have nominal size with paired, binary-outcome, group randomized trials
• PERMUTATION TESTS AND OTHER TEST STATISTICS FOR ILL-BEHAVED DATA: EXPERIENCE OF THE NINDS t-PA STROKE TRIAL
• On the weak consistency of permutation tests
• Learning Theory
• The use of permutation tests for the analysis of parallel and stepped-wedge cluster-randomized trials
• Two applications of permutation tests in biostatistics
• Bootstrapping and permuting paired \(t\)-test type statistics

This page was built for publication: Invalid permutation tests

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q978997)