A framework for Monte Carlo based multiple testing
From MaRDI portal
Abstract: We are concerned with a situation in which we would like to test multiple hypotheses with tests whose p-values cannot be computed explicitly but can be approximated using Monte Carlo simulation. This scenario occurs widely in practice. We are interested in obtaining the same rejections and non-rejections as the ones obtained if the p-values for all hypotheses had been available. The present article introduces a framework for this scenario by providing a generic algorithm for a general multiple testing procedure. We establish conditions which guarantee that the rejections and non-rejections obtained through Monte Carlo simulations are identical to the ones obtained with the p-values. Our framework is applicable to a general class of step-up and step-down procedures which includes many established multiple testing corrections such as the ones of Bonferroni, Holm, Sidak, Hochberg or Benjamini-Hochberg. Moreover, we show how to use our framework to improve algorithms available in the literature in such a way as to yield theoretical guarantees on their results. These modifications can easily be implemented in practice and lead to a particular way of reporting multiple testing results as three sets together with an error bound on their correctness, demonstrated exemplarily using a real biological dataset.
Recommendations
- MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests
- A simple method for implementing Monte Carlo tests
- QuickMMCTest: quick multiple Monte Carlo testing
- Finite-sample inference and nonstandard asymptotics with Monte Carlo tests and R
- Optimal allocation of Monte Carlo simulations to multiple hypothesis tests
Cites work
- A Bernstein-type estimator for decreasing density with application to p-value adjustments
- A Direct Approach to False Discovery Rates
- A sharper Bonferroni procedure for multiple tests of significance
- A stagewise rejective multiple test procedure based on a modified Bonferroni test
- A test for partial differential expression
- Adaptive choice of the number of bootstrap samples in large scale multiple testing
- An improved Bonferroni procedure for multiple tests of significance
- Estimating the Proportion of True Null Hypotheses, with application to DNA Microarray Data
- Estimation of the proportion of true null hypotheses in high-dimensional data under dependence
- False Discovery Control for Multiple Tests of Association Under General Dependence
- scientific article; zbMATH DE number 5361932 (Why is no real title available?)
- scientific article; zbMATH DE number 3624650 (Why is no real title available?)
- scientific article; zbMATH DE number 720689 (Why is no real title available?)
- scientific article; zbMATH DE number 3026527 (Why is no real title available?)
- Internal validation inferences of significant genomic features in genome-wide screening
- MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests
- Modified Sequentially Rejective Multiple Test Procedures
- Multiple comparison procedures for discrete test statistics.
- On confidence sequences
- On weighted Hochberg procedures
- Probability Inequalities for Sums of Bounded Random Variables
- Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
- Statistical properties of an early stopping rule for resampling-based multiple testing
- Stepup procedures for control of generalizations of the familywise error rate
- The control of the false discovery rate in multiple testing under dependency.
- TreeHugger: a new test for enrichment of gene ontology terms
Cited in
(15)- Markov chain Monte Carlo tests for designed experiments
- On the expected runtime of multiple testing algorithms with bounded error
- A simple method for implementing Monte Carlo tests
- Optimal allocation of Monte Carlo simulations to multiple hypothesis tests
- Closure properties of classes of multiple testing procedures
- QuickMMCTest: quick multiple Monte Carlo testing
- Monte Carlo null models for genomic data
- MMCTest -- a safe algorithm for implementing multiple Monte Carlo tests
- Monte Carlo study of forward stepwise discrimination based on small samples
- Tests based on monte carlo simulations conditioned on maximum likelihood estimates of nuisance parameters
- Implementing Monte Carlo tests with \(p\)-value buckets
- Likelihood, Replicability and Robbins' Confidence Sequences
- MERIT: controlling Monte-Carlo error rate in large-scale Monte-Carlo hypothesis testing
- Multiple testing with anytime-valid Monte Carlo p-values
- Multiple Monte Carlo testing, with applications in spatial point processes
This page was built for publication: A framework for Monte Carlo based multiple testing
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2835309)
