Generalizations of the familywise error rate

DOI10.1214/009053605000000084zbMATH Open1072.62060arXivmath/0507420OpenAlexW2952085197MaRDI QIDQ2569237FDOQ2569237

Authors: Joseph P. Romano, Erich L. Lehmann 

Publication date: 18 October 2005

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Consider the problem of simultaneously testing null hypotheses H_1,...,H_s. The usual approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, particularly if s is large, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which we call the k-FWER. We derive both single-step and stepdown procedures that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. In particular, we derive a stepdown procedure that is quite simple to apply, and prove that it cannot be improved without violation of control of the k-FWER. We also consider the false discovery proportion (FDP) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289-300] controls E(FDP). Here, we construct methods such that, for any gamma and alpha, P{FDP>gamma}lealpha. Two stepdown methods are proposed. The first holds under mild conditions on the dependence structure of p-values, while the second is more conservative but holds without any dependence assumptions.

Full work available at URL: https://arxiv.org/abs/math/0507420

Recommendations

• Stepup procedures for control of generalizations of the familywise error rate
• Control of generalized error rates in multiple testing
• On stepdown control of the false discovery proportion
• Step-up and step-down procedures controlling the number and proportion of false positives
• Stepup procedures controlling generalized FWER and generalized FDR

zbMATH Keywords

multiple testing\(p\)-valuesstepdown procedures

Mathematics Subject Classification ID

Nonparametric hypothesis testing (62G10) Paired and multiple comparisons; multiple testing (62J15)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• An improved Bonferroni procedure for multiple tests of significance
• The control of the false discovery rate in multiple testing under dependency.
• A stochastic process approach to false discovery control.
• The Simes Method for Multiple Hypothesis Testing With Positively Dependent Test Statistics
• False Discovery Control for Random Fields
• Controlling the number of false discoveries: application to high-dimensional genomic data
• Title not available (Why is that?)
• Title not available (Why is that?)
• Some probability inequalities for ordered MTP\(_2\) random variables: a proof of the Simes conjecture
• Title not available (Why is that?)
• Multiple testing of pairs of one-sided hypotheses

Cited In (only showing first 100 items - show all)

• Bonferroni-type Plug-in Procedure Controlling Generalized Familywise Error Rate
• Control of the false discovery rate under dependence using the bootstrap and subsampling
• A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion
• Comparing distributions by multiple testing across quantiles or CDF values
• Strong approximations of level exceedences related to multiple hypothesis testing
• New multiple testing method under no dependency assumption, with application to multiple comparisons problem
• Control of the false discovery proportion for independently tested null hypotheses
• Microarrays, empirical Bayes and the two-groups model
• Generalized Augmentation to Control the False Discovery Exceedance in Multiple Testing
• Parametric versus nonparametrics: two alternative methodologies
• Asymptotic Bayes-optimality under sparsity of some multiple testing procedures
• Familywise error rate control via knockoffs
• Step-up and step-down methods for testing multiple hypotheses in sequential experiments
• Asymptotic properties of false discovery rate controlling procedures under independence
• Inequalities between generalized familywise error rates of a multiple testing procedure
• On stepdown control of the false discovery proportion
• Copulas, uncertainty, and false discovery rate control
• On stepwise control of the generalized familywise error rate
• Step-up procedure controlling generalized family-wise error rate
• On a generalized false discovery rate
• Exact calculations for false discovery proportion with application to least favorable configura\-tions
• Bound on FWER for correlated normal
• Multiple hypothesis testing on composite nulls using constrained \(p\)-values
• A note on control of the false discovery proportion
• Statistical procedures for the market graph construction
• A sharp upper bound for the expected number of false rejections
• Generalizing Simes' test and Hochberg's stepup procedure
• A novel characterization of the generalized family wise error rate using empirical null distributions
• Simultaneous and selective inference: Current successes and future challenges
• Deriving and comparing the distribution for the number of false positives in single step methods to control \(k\)-FWER
• Optimality of the Holm procedure among general step-down multiple testing procedures
• Confidence-based reasoning in stochastic constraint programming
• A Conservative Test for Multiple Comparison Based on Highly Correlated Test Statistics
• On the false discovery proportion convergence under Gaussian equi-correlation
• New procedures controlling the false discovery proportion via Romano-Wolf's heuristic
• Estimation of empirical null using a mixture of normals and its use in local false discovery rate
• Power, FDR and conservativeness of BB-SGoF method
• Power-enhanced multiple decision functions controlling family-wise error and false discovery rates
• Stepup procedures controlling generalized FWER and generalized FDR
• Further results on controlling the false discovery proportion
• Simultaneous critical values for \(t\)-tests in very high dimensions
• A Note on Controlling the Number of False Positives
• Type I error rate control for testing many hypotheses: a survey with proofs
• Empirical Likelihood for the Analysis of Experimental Designs
• Multiple testing and error control in Gaussian graphical model selection
• Multiple Comparisons Controlling Expected Number of False Discoveries
• On correlated \(z\)-values distribution in hypothesis testing
• Exceedance Control of the False Discovery Proportion
• Step-up and step-down procedures controlling the number and proportion of false positives
• Some optimality properties of FDR controlling rules under sparsity
• Control of generalized error rates in multiple testing
• FORMALIZED DATA SNOOPING BASED ON GENERALIZED ERROR RATES
• Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures
• Balanced control of generalized error rates
• Two simple sufficient conditions for FDR control
• k-FWER control without \(p \)-value adjustment, with application to detection of genetic determinants of multiple sclerosis in Italian twins
• Some Results on the Control of the False Discovery Rate under Dependence
• Robustness of multiple testing procedures against dependence
• A new multiple testing method in the dependent case
• Rejoinder on: Control of the false discovery rate under dependence using the bootstrap and subsampling
• Multiple testing via \(\mathrm{FDR}_L\) for large-scale imaging data
• On improving some adaptive BH procedures controlling the FDR under dependence
• Error rates for multivariate outlier detection
• Stepup procedures for control of generalizations of the familywise error rate
• Farmtest: factor-adjusted robust multiple testing with approximate false discovery control
• Controlling the false discovery rate with constraints: the Newman-Keuls test revisited
• A comprehensive error rate for multiple testing
• Competition-based control of the false discovery proportion
• FWER goes to zero for correlated normal
• Uncertainty quantification for robust variable selection and multiple testing
• Sequential tests controlling generalized familywise error rates
• Measuring network systemic risk contributions: a leave-one-out approach
• Conditional calibration for false discovery rate control under dependence
• Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings
• A new perspective on robust \(M\)-estimation: finite sample theory and applications to dependence-adjusted multiple testing
• On hypothesis testing with a partitioned random alternative
• Bounds on generalized family-wise error rates for normal distributions
• Rejoinder to discussions on ``Optimal test procedures for multiple hypotheses controlling the familywise expected loss
• Online rules for control of false discovery rate and false discovery exceedance
• Classes of multiple decision functions strongly controlling FWER and FDR
• New testing procedures with \(k\)-FWER control for discrete data
• Stepdown Procedures Controlling a Generalized False Discovery Rate
• Multiple testing approaches for hypotheses in integrative genomics
• A sufficient criterion for control of some generalized error rates in multiple testing
• High-dimensional mean estimation via \(\ell_1\) penalized normal likelihood
• General theory of mixture procedures for gatekeeping
• Note on a stepwise procedure for rejecting at least k out of m hypotheses: A simple Holm‐type formulation and proof
• Efficient signal inclusion with genomic applications
• Procedures controlling the \(k\)-FDR using bivariate distributions of the null \(p\)-values
• An Empirical Bayes Approach to Controlling the False Discovery Exceedance
• Only closed testing procedures are admissible for controlling false discovery proportions
• Simultaneous testing of multiple hypotheses using generalized \(p\)-values
• Asymptotically optimal sequential multiple testing with asynchronous decisions
• Some implications of an alternative definition of the multiple comparison problem
• Simultaneous confidence regions for closed tests, including Holm-, Hochberg-, and Hommel-related procedures
• Flexible Regression Model Selection for Survival Probabilities: With Application to AIDS
• Family-wise error rate of a step-down procedure
• Controlling the false discovery exceedance for heterogeneous tests
• Variable selection using stepdown procedures in high-dimensional linear models
• The false discovery rate (FDR) of multiple tests in a class room lecture

This page was built for publication: Generalizations of the familywise error rate

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2569237)