A simple method for implementing Monte Carlo tests
From MaRDI portal
Abstract: We consider a statistical test whose p-value can only be approximated using Monte Carlo simulations. We are interested in deciding whether the p-value for an observed data set lies above or below a given threshold such as 5%. We want to ensure that the resampling risk, the probability of the (Monte Carlo) decision being different from the true decision, is uniformly bounded. This article introduces a simple open-ended method with this property, the confidence sequence method (CSM). We compare our approach to another algorithm, SIMCTEST, which also guarantees an (asymptotic) uniform bound on the resampling risk, as well as to other Monte Carlo procedures without a uniform bound. CSM is free of tuning parameters and conservative. It has the same theoretical guarantee as SIMCTEST and, in many settings, similar stopping boundaries. As it is much simpler than other methods, CSM is a useful method for practical applications.
Recommendations
- Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk
- A framework for Monte Carlo based multiple testing
- Implementing Monte Carlo tests with \(p\)-value buckets
- MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests
- Power of the Sequential Monte Carlo Test
Cites work
- A framework for Monte Carlo based multiple testing
- An algorithm to compute the power of Monte Carlo tests with guaranteed precision
- Bootstrap tests: how many bootstraps?
- Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests
- Designing Monte Carlo implementations of permutation or bootstrap hypothesis tests
- scientific article; zbMATH DE number 3810685 (Why is no real title available?)
- scientific article; zbMATH DE number 1104922 (Why is no real title available?)
- scientific article; zbMATH DE number 3026527 (Why is no real title available?)
- scientific article; zbMATH DE number 3046453 (Why is no real title available?)
- MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests
- On confidence sequences
- Optimal generalized truncated sequential Monte Carlo test
- Power of the Sequential Monte Carlo Test
- Prospective Time Periodic Geographical Disease Surveillance Using a Scan Statistic
- Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk
- Sequential Tests of Statistical Hypotheses
- Statistical Methods Related to the Law of the Iterated Logarithm
- Truncated sequential Monte Carlo test with exact power
- “Generalization of the durbin-watson statistic for higher order autoregressive processes
Cited in
(12)- An algorithm to compute the power of Monte Carlo tests with guaranteed precision
- QuickMMCTest: quick multiple Monte Carlo testing
- Monte Carlo null models for genomic data
- A framework for Monte Carlo based multiple testing
- Computing critical values of exact tests by incorporating Monte Carlo simulations combined with statistical tables
- MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests
- Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk
- Power of the Sequential Monte Carlo Test
- Implementing Monte Carlo tests with \(p\)-value buckets
- Designing Monte Carlo implementations of permutation or bootstrap hypothesis tests
- Enhancing Performance of Random Testing through Markov Chain Monte Carlo Methods
- Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests
This page was built for publication: A simple method for implementing Monte Carlo tests
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2203425)
