Permutation tests using arbitrary permutation distributions
From MaRDI portal
Publication:6112552
Abstract: Permutation tests date back nearly a century to Fisher's randomized experiments, and remain an immensely popular statistical tool, used for testing hypotheses of independence between variables and other common inferential questions. Much of the existing literature has emphasized that, for the permutation p-value to be valid, one must first pick a subgroup of permutations (which could equal the full group) and then recalculate the test statistic on permuted data using either an exhaustive enumeration of , or a sample from drawn uniformly at random. In this work, we demonstrate that the focus on subgroups and uniform sampling are both unnecessary for validity -- in fact, a simple random modification of the permutation p-value remains valid even when using an arbitrary distribution (not necessarily uniform) over any subset of permutations (not necessarily a subgroup). We provide a unified theoretical treatment of such generalized permutation tests, recovering all known results from the literature as special cases. Thus, this work expands the flexibility of the permutation test toolkit available to the practitioner.
Cites work
- Another Look at the Lady Tasting Tea and Differences Between Permutation Tests and Randomisation Tests
- Combining p-values via averaging
- Conformal prediction beyond exchangeability
- Conservative hypothesis tests and confidence intervals using importance sampling
- Consistency of invariance-based randomization tests
- Exact testing with random permutations
- Generalized Monte Carlo significance tests
- scientific article; zbMATH DE number 2168212 (Why is no real title available?)
- scientific article; zbMATH DE number 7625196 (Why is no real title available?)
- Minimax optimality of permutation tests
- Random variables with maximum sums
- Testing Statistical Hypotheses
Cited in
(22)- Permutation tests. A practical guide to resampling methods for testing hypotheses
- Robust permutation tests for one sample
- Permutation tests for general dependent truncation
- Permutation tests. A practical guide to resampling methods for testing hypotheses.
- Exact testing with random permutations
- Some Elementary Theory of Permutation Tests
- On Inverting Permutation Tests
- Densities in large permutations and parameter testing
- Transformations which preserve exchangeability and application to permutation tests
- Permutation tests for homogeneity based on some characterizations
- An adaptive permutation test procedure for several common tests of significance.
- Permutation Tests at Nonparametric Rates
- Hoeffding and Bernstein inequalities for weighted sums of exchangeable random variables
- A geometrically informed permutation test for dependency in spatiotemporal patterns of protein species in microscopy images
- Markov chain Monte Carlo significance tests
- Randomization tests for conditional group symmetry
- Power enhancement of permutation-augmented partial-correlation tests via fixed-row permutations
- SymmPI: predictive inference for data with group symmetries
- On the Term “Randomization Test”
- Fast Conservative Monte Carlo Confidence Sets
- Measuring evidence against exchangeability and group invariance with e-values
- An MCMC hypothesis test to check a claimed sampler: applied to a claimed sampler for the G-Wishart distribution
This page was built for publication: Permutation tests using arbitrary permutation distributions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6112552)
