Stepup procedures for control of generalizations of the familywise error rate

DOI10.1214/009053606000000461MaRDI QIDQ449953zbMATH OpenOpenAlexFDO

Authors Joseph P. Romano, Azeem M. Shaikh

Publication date 3 September 2012

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/math/0611266, https://projecteuclid.org/euclid.aos/1162567636

zbMATH Keywords

false discovery ratemultiple testingfalse discovery proportionstepdown procedurep-value

Mathematics Subject Classification ID

Paired and multiple comparisons; multiple testing (62J15)

Abstract: Consider the multiple testing problem of testing null hypotheses $H_1,\mathrm{...},H_s$. A classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate ($mathitFWER$), the probability of even one false rejection. But if $s$ is large, control of the $mathitFWER$ is so stringent that the ability of a procedure that controls the $mathitFWER$ to detect false null hypotheses is limited. It is therefore desirable to consider other measures of error control. This article considers two generalizations of the $mathitFWER$. The first is the $k-mathitFWER$, in which one is willing to tolerate $k$ or more false rejections for some fixed $kgeq1$. The second is based on the false discovery proportion ($mathitFDP$), defined to be the number of false rejections divided by the total number of rejections (and defined to be 0 if there are no rejections). Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289--300] proposed control of the false discovery rate ($mathitFDR$), by which they meant that, for fixed $alpha$, $E(mathitFDP)leqalpha$. Here, we consider control of the $mathitFDP$ in the sense that, for fixed $gamma$ and $alpha$, $PmathitFDP>gammaleqalpha$. Beginning with any nondecreasing sequence of constants and $p$-values for the individual tests, we derive stepup procedures that control each of these two measures of error control without imposing any assumptions on the dependence structure of the $p$-values. We use our results to point out a few interesting connections with some closely related stepdown procedures. We then compare and contrast two $mathitFDP$-controlling procedures obtained using our results with the stepup procedure for control of the $mathitFDR$ of Benjamini and Yekutieli [Ann. Statist. 29 (2001) 1165--1188].

Recommendations

• Generalizations of the familywise error rate
• On stepdown control of the false discovery proportion
• Step-up and step-down procedures controlling the number and proportion of false positives
• Control of generalized error rates in multiple testing
• Stepup procedures controlling generalized FWER and generalized FDR

Cites work

• scientific article; zbMATH DE number 3831073 (Why is no real title available?)
• scientific article; zbMATH DE number 3624650 (Why is no real title available?)
• scientific article; zbMATH DE number 720689 (Why is no real title available?)
• scientific article; zbMATH DE number 2220287 (Why is no real title available?)
• A sharper Bonferroni procedure for multiple tests of significance
• A stochastic process approach to false discovery control.
• Asymptotic comparison of step-down and step-up multiple test procedures based on exchangeable test statistics
• Control of generalized error rates in multiple testing
• Controlling the number of false discoveries: application to high-dimensional genomic data
• Generalizations of the familywise error rate
• On stepdown control of the false discovery proportion
• Stepwise Multiple Testing as Formalized Data Snooping
• Testing Statistical Hypotheses
• The Simes Method for Multiple Hypothesis Testing With Positively Dependent Test Statistics
• The control of the false discovery rate in multiple testing under dependency.

Cited in

(67)

• Graphical approaches for the control of generalized error rates
• On control of the false discovery rate under no assumption of dependency
• A comprehensive error rate for multiple testing
• Bonferroni-type Plug-in Procedure Controlling Generalized Familywise Error Rate
• Control of the false discovery rate under dependence using the bootstrap and subsampling
• Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates
• Controlling the false discovery rate with constraints: the Newman-Keuls test revisited
• On two results in multiple testing
• Control of the false discovery proportion for independently tested null hypotheses
• Testing the correlated random coefficient model
• Familywise error rate control via knockoffs
• A framework for Monte Carlo based multiple testing
• scientific article; zbMATH DE number 2220286 (Why is no real title available?)
• On least favorable configurations for step-up-down tests
• A Generalized Sidak-Holm Procedure and Control of Generalized Error Rates under Independence
• Some results on false discovery rate in stepwise multiple testing procedures.
• Asymptotic properties of false discovery rate controlling procedures under independence
• Inequalities between generalized familywise error rates of a multiple testing procedure
• Multiple hypotheses testing and expected number of type I errors
• Generalizations of the familywise error rate
• On stepdown control of the false discovery proportion
• Step-up procedure controlling generalized family-wise error rate
• On stepwise control of the generalized familywise error rate
• On a generalized false discovery rate
• Decision theory results for one-sided multiple comparison procedures
• Exact calculations for false discovery proportion with application to least favorable configura\-tions
• Classes of multiple decision functions strongly controlling FWER and FDR
• New testing procedures with \(k\)-FWER control for discrete data
• Stepdown Procedures Controlling a Generalized False Discovery Rate
• Dealing with imperfect randomization: inference for the highscope Perry preschool program
• A note on control of the false discovery proportion
• A sufficient criterion for control of some generalized error rates in multiple testing
• A sharp upper bound for the expected number of false rejections
• Generalizing Simes' test and Hochberg's stepup procedure
• Closure properties of classes of multiple testing procedures
• Procedures controlling the \(k\)-FDR using bivariate distributions of the null \(p\)-values
• Only closed testing procedures are admissible for controlling false discovery proportions
• Simultaneous testing of multiple hypotheses using generalized \(p\)-values
• Two-stage stepup procedures controlling FDR
• A Conservative Test for Multiple Comparison Based on Highly Correlated Test Statistics
• Controlling false discovery proportion in identification of drug-related adverse events from multiple system organ classes
• New procedures controlling the false discovery proportion via Romano-Wolf's heuristic
• Stepup procedures controlling generalized FWER and generalized FDR
• Flexible Regression Model Selection for Survival Probabilities: With Application to AIDS
• Further results on controlling the false discovery proportion
• Family-wise error rate of a step-down procedure
• Multiple comparisons with control under stochastic ordering: controlling FDR
• Parsimonious covariate selection with censored outcomes
• A note on multiple testing for composite null hypotheses
• On stepwise control of directional errors under independence and some dependence
• Multiple Comparisons Controlling Expected Number of False Discoveries
• Type I error rate control for testing many hypotheses: a survey with proofs
• On the False Discovery Rate and Expected Type I Errors
• Family-wise separation rates for multiple testing
• Improved power of familywise error rate procedures for discrete data under dependency
• Step-up and step-down procedures controlling the number and proportion of false positives
• Control of generalized error rates in multiple testing
• Balanced control of generalized error rates
• Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures
• Weighted multiple testing procedure for grouped hypotheses with \(k\)-FWER control
• FORMALIZED DATA SNOOPING BASED ON GENERALIZED ERROR RATES
• Two simple sufficient conditions for FDR control
• False discovery and false nondiscovery rates in single-step multiple testing procedures
• Student-Newman-Kuels controls the false discovery rate
• A comment on: ``Invidious comparisons: ranking and selection as compound decisions by Jiaying Gu and Roger Koenker
• scientific article; zbMATH DE number 2220287 (Why is no real title available?)
• A generalized family of multiple test procedures

This page was built for publication: Stepup procedures for control of generalizations of the familywise error rate

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q449953)