Stepup procedures for control of generalizations of the familywise error rate

DOI10.1214/009053606000000461zbMATH Open1246.62172OpenAlexW3098682883MaRDI QIDQ449953FDOQ449953

Authors Joseph P. Romano, Azeem M. Shaikh

Publication date 3 September 2012

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/math/0611266

zbMATH Keywords

false discovery ratemultiple testingfalse discovery proportionstepdown procedurep-value

Mathematics Subject Classification ID

Paired and multiple comparisons; multiple testing (62J15)

Abstract: Consider the multiple testing problem of testing null hypotheses $H_1,\mathrm{...},H_s$. A classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate ($mathitFWER$), the probability of even one false rejection. But if $s$ is large, control of the $mathitFWER$ is so stringent that the ability of a procedure that controls the $mathitFWER$ to detect false null hypotheses is limited. It is therefore desirable to consider other measures of error control. This article considers two generalizations of the $mathitFWER$. The first is the $k-mathitFWER$, in which one is willing to tolerate $k$ or more false rejections for some fixed $kgeq1$. The second is based on the false discovery proportion ($mathitFDP$), defined to be the number of false rejections divided by the total number of rejections (and defined to be 0 if there are no rejections). Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289--300] proposed control of the false discovery rate ($mathitFDR$), by which they meant that, for fixed $alpha$, $E(mathitFDP)leqalpha$. Here, we consider control of the $mathitFDP$ in the sense that, for fixed $gamma$ and $alpha$, $PmathitFDP>gammaleqalpha$. Beginning with any nondecreasing sequence of constants and $p$-values for the individual tests, we derive stepup procedures that control each of these two measures of error control without imposing any assumptions on the dependence structure of the $p$-values. We use our results to point out a few interesting connections with some closely related stepdown procedures. We then compare and contrast two $mathitFDP$-controlling procedures obtained using our results with the stepup procedure for control of the $mathitFDR$ of Benjamini and Yekutieli [Ann. Statist. 29 (2001) 1165--1188].

Recommendations

• Generalizations of the familywise error rate
• On stepdown control of the false discovery proportion
• Step-up and step-down procedures controlling the number and proportion of false positives
• Control of generalized error rates in multiple testing
• Stepup procedures controlling generalized FWER and generalized FDR

Cites work

• scientific article; zbMATH DE number 3831073 (Why is no real title available?)
• scientific article; zbMATH DE number 3624650 (Why is no real title available?)
• scientific article; zbMATH DE number 720689 (Why is no real title available?)
• scientific article; zbMATH DE number 2220287 (Why is no real title available?)
• A sharper Bonferroni procedure for multiple tests of significance
• A stochastic process approach to false discovery control.
• Asymptotic comparison of step-down and step-up multiple test procedures based on exchangeable test statistics
• Control of generalized error rates in multiple testing
• Controlling the number of false discoveries: application to high-dimensional genomic data
• Generalizations of the familywise error rate
• On stepdown control of the false discovery proportion
• Stepwise Multiple Testing as Formalized Data Snooping
• Testing Statistical Hypotheses
• The Simes Method for Multiple Hypothesis Testing With Positively Dependent Test Statistics
• The control of the false discovery rate in multiple testing under dependency.

Cited in

(67)

• Dealing with imperfect randomization: inference for the highscope Perry preschool program
• A comment on: ``Invidious comparisons: ranking and selection as compound decisions by Jiaying Gu and Roger Koenker
• Parsimonious covariate selection with censored outcomes
• Stepdown Procedures Controlling a Generalized False Discovery Rate
• Controlling false discovery proportion in identification of drug-related adverse events from multiple system organ classes
• Graphical approaches for the control of generalized error rates
• On stepwise control of the generalized familywise error rate
• Classes of multiple decision functions strongly controlling FWER and FDR
• Further results on controlling the false discovery proportion
• On stepwise control of directional errors under independence and some dependence
• Only closed testing procedures are admissible for controlling false discovery proportions
• Familywise error rate control via knockoffs
• Controlling the false discovery rate with constraints: the Newman-Keuls test revisited
• Family-wise error rate of a step-down procedure
• A generalized family of multiple test procedures
• Closure properties of classes of multiple testing procedures
• Some results on false discovery rate in stepwise multiple testing procedures.
• Generalizations of the familywise error rate
• Step-up procedure controlling generalized family-wise error rate
• A comprehensive error rate for multiple testing
• A note on multiple testing for composite null hypotheses
• Multiple hypotheses testing and expected number of type I errors
• Testing the correlated random coefficient model
• Procedures controlling the \(k\)-FDR using bivariate distributions of the null \(p\)-values
• On control of the false discovery rate under no assumption of dependency
• Bonferroni-type Plug-in Procedure Controlling Generalized Familywise Error Rate
• Decision theory results for one-sided multiple comparison procedures
• New testing procedures with \(k\)-FWER control for discrete data
• Type I error rate control for testing many hypotheses: a survey with proofs
• On a generalized false discovery rate
• Control of generalized error rates in multiple testing
• A sufficient criterion for control of some generalized error rates in multiple testing
• FORMALIZED DATA SNOOPING BASED ON GENERALIZED ERROR RATES
• On the False Discovery Rate and Expected Type I Errors
• Flexible Regression Model Selection for Survival Probabilities: With Application to AIDS
• Inequalities between generalized familywise error rates of a multiple testing procedure
• A Generalized Sidak-Holm Procedure and Control of Generalized Error Rates under Independence
• False discovery and false nondiscovery rates in single-step multiple testing procedures
• Explicit formulas for generalized family-wise error rates and unimprovable step-down multiple testing procedures
• Family-wise separation rates for multiple testing
• Generalizing Simes' test and Hochberg's stepup procedure
• Simultaneous testing of multiple hypotheses using generalized \(p\)-values
• Weighted multiple testing procedure for grouped hypotheses with \(k\)-FWER control
• Step-up and step-down procedures controlling the number and proportion of false positives
• Two-stage stepup procedures controlling FDR
• Multiple Comparisons Controlling Expected Number of False Discoveries
• scientific article; zbMATH DE number 2220286 (Why is no real title available?)
• On least favorable configurations for step-up-down tests
• Student-Newman-Kuels controls the false discovery rate
• Control of the false discovery rate under dependence using the bootstrap and subsampling
• Exact calculations for false discovery proportion with application to least favorable configura\-tions
• New procedures controlling the false discovery proportion via Romano-Wolf's heuristic
• Balanced control of generalized error rates
• On two results in multiple testing
• On stepdown control of the false discovery proportion
• A note on control of the false discovery proportion
• Improved power of familywise error rate procedures for discrete data under dependency
• A framework for Monte Carlo based multiple testing
• A sharp upper bound for the expected number of false rejections
• Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates
• Control of the false discovery proportion for independently tested null hypotheses
• Two simple sufficient conditions for FDR control
• A Conservative Test for Multiple Comparison Based on Highly Correlated Test Statistics
• Multiple comparisons with control under stochastic ordering: controlling FDR
• scientific article; zbMATH DE number 2220287 (Why is no real title available?)
• Asymptotic properties of false discovery rate controlling procedures under independence
• Stepup procedures controlling generalized FWER and generalized FDR

This page was built for publication: Stepup procedures for control of generalizations of the familywise error rate

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q449953)