The sequential rejection principle of familywise error control

DOI10.1214/10-AOS829zbMATH Open1204.62140arXiv1211.3313OpenAlexW2087651612MaRDI QIDQ620568FDOQ620568

Authors: Jelle J. Goeman, Aldo Solari 

Publication date: 19 January 2011

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Closed testing and partitioning are recognized as fundamental principles of familywise error control. In this paper, we argue that sequential rejection can be considered equally fundamental as a general principle of multiple testing. We present a general sequentially rejective multiple testing procedure and show that many well-known familywise error controlling methods can be constructed as special cases of this procedure, among which are the procedures of Holm, Shaffer and Hochberg, parallel and serial gatekeeping procedures, modern procedures for multiple testing in graphs, resampling-based multiple testing procedures and even the closed testing and partitioning procedures themselves. We also give a general proof that sequentially rejective multiple testing procedures strongly control the familywise error if they fulfill simple criteria of monotonicity of the critical values and a limited form of weak familywise error control in each single step. The sequential rejection principle gives a novel theoretical perspective on many well-known multiple testing procedures, emphasizing the sequential aspect. Its main practical usefulness is for the development of multiple testing procedures for null hypotheses, possibly logically related, that are structured in a graph. We illustrate this by presenting a uniform improvement of a recently published procedure.

Full work available at URL: https://arxiv.org/abs/1211.3313

Recommendations

• A rejection principle for sequential tests of multiple hypotheses controlling familywise error rates
• Sequentially rejective pairwise testing procedures
• On stepwise control of the generalized familywise error rate
• Modified Sequentially Rejective Multiple Test Procedures
• Sequential tests controlling generalized familywise error rates

zbMATH Keywords

graphmultiple testingfamilywise error rate

Mathematics Subject Classification ID

Applications of graph theory (05C90) Hypothesis testing in multivariate analysis (62H15) Sequential statistical analysis (62L10) Paired and multiple comparisons; multiple testing (62J15)

Cites Work

• Hierarchical testing of variable importance
• Title not available (Why is that?)
• An improved Bonferroni procedure for multiple tests of significance
• Testing Statistical Hypotheses
• Resampling-based multiple testing for microarray data analysis (With comments)
• A stagewise rejective multiple test procedure based on a modified Bonferroni test
• A sharper Bonferroni procedure for multiple tests of significance
• On closed testing procedures with special reference to ordered analysis of variance
• Title not available (Why is that?)
• Rectangular Confidence Regions for the Means of Multivariate Normal Distributions
• Optimally weighted, fixed sequence and gatekeeper multiple testing procedures
• Balanced control of generalized error rates
• Multiple testing procedures with applications to genomics.
• Testing hypotheses in order
• Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates
• Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing
• The partitioning principle: a powerful tool in multiple decision theory
• Modified Sequentially Rejective Multiple Test Procedures
• Some probability inequalities for ordered MTP\(_2\) random variables: a proof of the Simes conjecture
• Title not available (Why is that?)
• Bonferroni procedures for logically related hypotheses.
• Extensions of multiple testing procedures based on Simes' test
• An Improved Sequentially Rejective Bonferroni Test Procedure

Cited In (39)

• Multiple testing for exploratory research
• Goodness-of-Fit Testing for the Newcomb-Benford Law With Application to the Detection of Customs Fraud
• Hierarchical inference for genome-wide association studies: a view on methodology with software
• Shortcuts for locally consonant closed test procedures
• \(F\)-distribution calibrated empirical likelihood ratio tests for multiple hypothesis testing
• Discussion on “Testing small study effects in multivariate meta‐analysis” by Chuan Hong, Georgia Salanti, Sally Morton, Richard Riley, Haitao Chu, Stephen E. Kimmel and Yong Chen
• Multiple testing along a tree
• Assessing the pattern of covariance matrices via an augmentation multiple testing procedure
• A multiple testing method for hypotheses structured in a directed acyclic graph
• The union closure method for testing a fixed sequence of families of hypotheses
• Rejoinder
• Rotation-based multiple testing in the multivariate linear model
• Classes of multiple decision functions strongly controlling FWER and FDR
• Sequentially rejective pairwise testing procedures
• Detecting multiple replicating signals using adaptive filtering procedures
• Exact testing with random permutations
• A rejection principle for sequential tests of multiple hypotheses controlling familywise error rates
• Multiple testing approaches for hypotheses in integrative genomics
• Error control in tree structured hypothesis testing
• Discussion on “Adaptive enrichment designs with a continuous biomarker” by Nigel Stallard
• Rejoinder to discussion on “Adaptive enrichment designs with a continuous biomarker”
• Signal localization: a new approach in signal discovery
• Analyzing environmental-trait interactions in ecological communities with fourth-corner latent variable models
• Tests for Jumps in Yield Spreads
• Only closed testing procedures are admissible for controlling false discovery proportions
• Simultaneous confidence intervals for ranks using the partitioning principle
• A sequential rejection testing method for high-dimensional regression with correlated variables
• Order selection with confidence for finite mixture models
• An improved closed procedure for testing multiple hypotheses
• The partitioning principle: a powerful tool in multiple decision theory
• Rejoinder
• Minimally adaptive BH: a tiny but uniform improvement of the procedure of Benjamini and Hochberg
• Screening-assisted dynamic multiple testing with false discovery rate control
• On stepwise control of directional errors under independence and some dependence
• Asymptotic optimality of the Westfall-Young permutation procedure for multiple testing under dependence
• Applying the generalized partitioning principle to control the generalized familywise error rate
• A region-based multiple testing method for hypotheses ordered in space or time
• Optimal exact tests for multiple binary endpoints
• The online closure principle

This page was built for publication: The sequential rejection principle of familywise error control

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q620568)