Multiple testing for exploratory research
From MaRDI portal
Publication:5963767
Abstract: Motivated by the practice of exploratory research, we formulate an approach to multiple testing that reverses the conventional roles of the user and the multiple testing procedure. Traditionally, the user chooses the error criterion, and the procedure the resulting rejected set. Instead, we propose to let the user choose the rejected set freely, and to let the multiple testing procedure return a confidence statement on the number of false rejections incurred. In our approach, such confidence statements are simultaneous for all choices of the rejected set, so that post hoc selection of the rejected set does not compromise their validity. The proposed reversal of roles requires nothing more than a review of the familiar closed testing procedure, but with a focus on the non-consonant rejections that this procedure makes. We suggest several shortcuts to avoid the computational problems associated with closed testing.
Recommendations
- Correction note: ``Multiple testing for exploratory research
- scientific article; zbMATH DE number 3949560
- scientific article; zbMATH DE number 3905661
- Multistage tests of multiple hypotheses
- Multiple testing in a set of nested hypotheses
- Multiple Comparisons
- Multiple testing of causal hypotheses
- Multivariate Multiple Test Procedures
- Multiple testing for dose finding
Cites work
- scientific article; zbMATH DE number 3831073 (Why is no real title available?)
- scientific article; zbMATH DE number 720689 (Why is no real title available?)
- scientific article; zbMATH DE number 2220287 (Why is no real title available?)
- A Direct Approach to False Discovery Rates
- An improved Bonferroni procedure for multiple tests of significance
- Closed procedures are better and often admit a shortcut.
- Control of generalized error rates in multiple testing
- Empirical Bayes Analysis of a Microarray Experiment
- Estimating the Null and the Proportion of Nonnull Effects in Large-Scale Multiple Comparisons
- Estimating the Proportion of True Null Hypotheses, with application to DNA Microarray Data
- False Discovery Control for Multiple Tests of Association Under General Dependence
- False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters
- Hierarchical testing of variable importance
- Hochberg's Step-Up Method: Cutting Corners Off Holm's Step-Down Method
- Lower bounds for the number of false null hypotheses for multiple testing of associations under general dependence structures
- On closed testing procedures with special reference to ordered analysis of variance
- On the False Discovery Rate and Expected Type I Errors
- Optimal testing of multiple hypotheses with common effect direction
- Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
- Screening for Partial Conjunction Hypotheses
- Shortcuts for locally consonant closed test procedures
- Some probability inequalities for ordered MTP\(_2\) random variables: a proof of the Simes conjecture
- The control of the false discovery rate in multiple testing under dependency.
- The sequential rejection principle of familywise error control
Cited in
(39)- Hierarchical inference for genome-wide association studies: a view on methodology with software
- Logical and test consistency in pairwise multiple comparisons
- Exploratory inference: localizing relevant effects with confidence
- Combining partial true discovery guarantee procedures
- Adaptive novelty detection with false discovery rate guarantee
- Trade-off Between Validity and Efficiency of Merging P-Values Under Arbitrary Dependence
- Permutation-based true discovery proportions for functional magnetic resonance imaging cluster analysis
- Mediation effect selection in high-dimensional and compositional microbiome data
- A tutorial on statistically sound pattern discovery
- Only closed testing procedures are admissible for controlling false discovery proportions
- Exact testing with random permutations
- Controlling the false discovery exceedance for heterogeneous tests
- A region-based multiple testing method for hypotheses ordered in space or time
- Controlling costs: feature selection on a budget
- Semi-supervised multiple testing
- Exceedance control of the false discovery proportion via high precision inversion method of berk-Jones statistics
- Closed Testing with Globaltest, with Application in Metabolomics
- The online closure principle
- scientific article; zbMATH DE number 817577 (Why is no real title available?)
- On agnostic post-hoc approaches to false positive control
- Testing with \(\mathrm{p}^*\)-values: between p-values, mid p-values, and e-values
- Replicability across multiple studies
- A multiple testing method for hypotheses structured in a directed acyclic graph
- Admissible ways of merging \(p\)-values under arbitrary dependence
- E-values: calibration, combination and applications
- Filtering the Rejection Set While Preserving False Discovery Rate Control
- Testing partial conjunction hypotheses under dependency, with applications to meta-analysis
- \(F\)-distribution calibrated empirical likelihood ratio tests for multiple hypothesis testing
- Post-selection inference for e-value based confidence intervals
- True and false discoveries with independent and sequential \(e\)-values
- Post hoc confidence bounds on false positives using reference families
- Estimating minimum effect with outlier selection
- Hommel's procedure in linear time
- False discovery proportion estimation by permutations: confidence for significance analysis of microarrays
- Confidence and discoveries with \(e\)-values
- Discussion of ``Multiple testing for exploratory research by J. J. Goeman and A. Solari
- Discussion of ``Multiple testing for exploratory research by J. J. Goeman and A. Solari
- Large‐scale simultaneous inference under dependence
- Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings
This page was built for publication: Multiple testing for exploratory research
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5963767)
